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ABSTRACT 



This document contains 14 research paper presentations and 5 
"mini- tips" from the 1998 American Vocational Education Research Association 
(AVERA) annual meeting. The first section includes three papers on 
international and distance education: "Determining Success of Vocational 
Students Enrolled in Distance Education Courses" (Michael K. Swan, Diane H. 
Jackman) ; "The Use of Applied Communications in Developing Critical Thinking 
Skills of Tech Prep Students" (Don R. Gelven, Bob R. Stewart) ; and "Using 
Delphi Technique to Create a Definition of Internationalization" (Barbara G. 
Ludwig) . Three papers on school -to- work transition issues are included in the 
second section: "School-to-Work Curricula in the Middle Schools: Benefits, 
Issues, and Concerns" (Curtis Finch, Marianne Mooney) ; "Comparing Three 
Automotive Partnership Programs at the Community College Level with the 
School-to-Work Model" (Michael L Klyde, Leonard Albright) ; and "Stages of 
Concern of Administrators and Teachers in the Implementation of the School to 
Work Transition Initiative in North Carolina" (Barbara M. Kirby, Wilbur 
Smith) . Two papers on imiversity instruction in agriculture make up the third 
section: "Attitudes of College of Agriculture Freshmen Toward Agriculture" 
(Lisa Breja, James E. Dyer, Randall J. Andreasen) ; and "Safety Issues in 
Agriculture Education Laboratories" (James E. Dyer and Randall Andreasen) . 

The fourth section contains papers on recent doctoral research in vocational 
education: "Factors Influencing Females Choosing Nontraditional 
Vocational-Technical Occupations" (Susan Deege) ; "Work Values and Leadership 
Styles of Manufacturing Employees in the U.S. and Germany: A Cross Cultural 
Comparison" (K. Peter Kuchinke) ; "Technical and Occupational Education 
Instructors ' Perceived Uses of Student Assessment Information in Making 
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Educational Decisions” (Greg Belcher) ; and ”The Education of Retail Managers 
through Management Training Programs in Apparel Retail Organizations” (Irene 
M. Foster, William L. Theummel) . In the fifth section, two papers explore 
assessment in vocational education: "Assessment, of Work-Based Learning 
Programs in Georgia” (Clifton Smith) ; and "Assessment in the Vocational 
Education Classroom: Using Information for Instructional Decisions” (Howard 
R. D. Gordon, Richard J. Yocke) . Following the papers are "mini-tips” on 
issues in vocational educational research: "Using Qualitative Research to 
Frame the Examination of an Alternative Vocational Teacher Education Program" 
(Gloria Heberly) ; "Characteristics of a Conceptual Framework Addressing the 
Environment of Instructional Space” (Lennie Scott-Webber) ; "Strengths and 
Weaknesses of E-mail as a Survey Research Method" (Allen D, Truell) ; "A 
Redesigned Program for the Preparation of Teachers” (Bob R, Stewart, Martin 
Bergee) ; and "Exploratory Style of Vocational Educators" (Helen C. Hall, 
Bettye P. Smith) . Individual papers contain references. (KC) 
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Notes from the Program Chair 



The 1998 American Vocational Education Research association (A VERA) 

Annual Meeting was held December 9-13, 1998 in New Orleans, Louisiana as part of the 
American Vocational Association (AVA) Annual Convention. The theme of the AVA 
Convention was “Creating Futures”. This Theme was carried over from the AVA 
Convention in 1997. 

AVERA is the research section of New and Related Services Division of AVA. 
Our organization had four research paper sessions, one doctoral dissertation paper 
session, one Grand Carousel section, the past presidents’ open forum and a business 
meeting. The complete program is presented following the Table of Contents in these 
Proceedings. The 28 papers submitted for consideration were all submitted to a panel of 
twelve reviewers in a blind peer-review process. Each submission was review by three 
reviewers. This process resulted in nineteen papers being accepted. All of the presenters 
provided their papers for inclusion in this set of Proceedings. 

I am grateful for all those individuals who provided their assistance and made the 
1998 annual research meeting at AVA possible. To the session chairs and facilitators and 
to the President, Wanda Stitt-Ghodes and Past-President, Doima Redmann, I owe my 
appreciation. I am also thankful to my program assistant, Monica Jackson, and graduate 
assistant, Herbie Barber, for their assistance with all of the meeting details and 
proceedings copy preparation. I am especially thankful for the assistance of the twelve 
reviewers of the proposals. These reviewers are: 

Kirby R. Barrick, University of Illinois, Urbana-Champaign; Wesley E. Budke, The Ohio 
State University; Michael F. Burnett, Louisiana State University; Carol A. Conroy, 
Cornell University; James Flowers, North Carolina State University; Maynard J. Iverson, 
University of Georgia; Theodore Lewis, University of Minnesota; Larry Miller, The Ohio 
State University; Doima H. Redmann, Louisiana State University; Wanda Stitt-Ghodes, 
The University of Georgia; Michael K. Swan, Washington State University; Dennis G. 
Tesolowski, Clemson University. 

It is my hope that the research presented here and the discussion that occurred at 
the meeting will advance both the profession and the individuals who participated. It has 
been my pleasure to serve as program chair but it was the assistance provided by those 
named above and many others that made this meeting and proceedings possible. 



Hollie B. Thomas For further information: 

AVERA President-Elect and Program Chair Hollie Thomas, Ph.D. 

Amercian Vocational Education Research Association Florida State University 
1998 AVERA/ AVA Annual Convention 113 Stone Building 

Tallahassee, Florida 32306 




6 



AGENDA 



NRS RESEARCH-DAY-BY-DAY LISTINGS 

*Wednesday, December 9, 1998* 

1:30 - 5:00 p.m 

Journal of Vocational Education Research Editorial Board 
Meeting 

Chair: Diane Jackman, Editorial Board Chair, North 

Dakota State University 

Facilitator: Jay Rojewski, JVER Editor, The University of 

Georgia, Athens, GA 



*Thursday, December 10, 1998* 

7:00- 10:00 a.m 

A VERA Executive Committee Breakfast Meeting (By 
Invitation Only) 

Chair: Wanda L. Stitt-Gohdes, AVERA President, The 

University of Georgia, Athens, GA 
Facilitator: Hollie B. Thomas, AVERA President-Elect, 

Florida State University, Tallahassee, FL 

1:30 -3:00 p.m 

International and Distance Education 



Topic 1: 
Speakers: 

Topic 2: 

Speakers: 
Topic 3: 
Speaker: 



Determining Success of Vocational Students 
Enrolled in Distance Education Courses 
Michael K. Swan, Washington State University, 
Diane H. Jackman, North Dakota State 
University 

The Use of Applied Communications in 
Developing Critical Thinking Skills of Tech 
Prep Students 

Don R. Gelven, Linn State Technical College, 
Bob R. Stewart, University of Missouri 
Using the Delphi Technique to Create a 
Definition of Internationalization 
Barbara G. Ludwig, Ohio State University 



Chair: Larry E. Miller, The Ohio State University 

Facilitator: Carol A. Conroy, Cornell University 
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3:30-5:00 p.m 
School -to — Work Issues 



Topic 1: 
Speakers: 



Topic 2 



Speakers: 
Topic 3: 



Speakers: 



School -to - Work Curricula in the Middle 
Schools: Benefits, Issues and Concerns 
Curtis Finch, Marianne Mooney, Virginia 
Polytechnic Institute, State University and 
National Center for Research in Vocational 
Education 

Comparing Three Automotive Partnership 

Programs at the Community College Level with 

the School -to - Work Model 

Michael L. Klyde, Leonard Albright, California 

State University at Long Beach 

Stages of Concern of Administrators and 

Teachers in the Implementation of the School - 

to - Work Transistion Initiative in North 

Carolina 

Barbara M. Kirby, North Carolina State 
University, Wilbur Smith, Jr, Public Schools of 
Robeson County, North Carolina 



Chair: Wesley E. Budke, The Ohio State University 

Facilitator: Marcia A. Anderson, Southern Illinois 

University 



*Friday, December 11, 1998* 

10:30 a.m. - 12:00 noon 

Critical Issues in Vocational Education: Open forum of 
A VERA Past Presidents 



Speakers: 

Chair: 

Facilitators: 



Past Presidents of the American Vocational 
Education Research Association (A VERA) 
Donna H. Redmann, Past President, Louisiana 
State University, Baton Rouge, LA 
Wanda L. Stitt-Gohdes, AVERA President, The 
Georgia State University, Athens, GA 
Hollie B. Thomas, AVERA President-Elect, 
Florida State University, Tallahassee, FL 
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12:00 noon - 1 :30 p.m. 
Past President Luncheon 



Chair: Donna H. Redmann, AVERA Past President, 

Louisiana State University, Baton Rouge, LA 
Facilitators: Wanda L. Stitt-Gohdes, AVERA President, The 
Georgia State University, Athens, GA 
Hollie B. Thomas, AVERA President-Elect, 
Florida State University, Tallahassee, FL 



1:30 - 5:00 p.m. 

Grand Carousel 

Issues in Vocational Education Research 



Carousel 1: 



Speaker: 
Carousel 2: 



Speaker: 
Carousel 3: 

Speaker: 

Carousel 4: 

Speaker: 
Carousel 5: 
Speaker: 



Using Qualitative Research to Frame the 
Examination of an Alternative Vocational 
Teacher Education Program 
Gloria Heberly, Temple University 
Characteristics of a Conceptual Framework 
Addressing the Environment of Instructional 
Space 

Lennie Scott-Webber, Virginia Tech 
Strengths and Weaknesses of E-mail as a Survey 
Research Method 

Allen D. Truell, University of Missouri- 
Columbia 

A Redesigned Program for the Preparation of 
Teachers 

Bob R. Stewart, University of Missouri 
Exploratory Style of Vocational Educators 
Bettye P. Smith, The University of Georgia 



5:30 - 7:30 p.m. 

AERA Presidential Address and Annual Business Meeting 

Chair: Donna H. Redmaim, Past President AVERA 

Louisiana State University, Baton Rouge, LA 
Facilitators: Wanda L. Stitt-Gohdes, AVERA President, The 
Georgia State University, Athens, GA 
Hollie B. Thomas, AVERA President-Elect, 
Florida State University, Tallahassee, FL 
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*Saturday, December 12, 1998* 



10:30 - 12:00 noon 

University Instruction in Agriculture 



Topic 1: 


Attitudes of Iowa State University College of 


Speakers: 


Agriculture Freshmen Toward Agriculture 
Randall Andreasen, Lisa Breja, James E. Dyer, 
Iowa State University 


Topic 2: 


Agriculture Education Laboratories: Are They 
Safe? 


Speakers: 


James E. Dyer, Randall Andreasen, Iowa State 
University 


Chair: 


Michael F. Burnett, Louisiana State University 



Facilitator: Joe W. Kotrlik, Louisiana State University 
1:30 - 3:00 p.m. 

Recent Doctoral Research in Vocational Education 



Topic 1: 


Factors Influencing Females Choosing 


Speaker: 
Topic 2: 


Nontraditional Vocational-Technical 
Occupations 

Susan Deege, Linden wood University 
Work Values and Leadership Styles of 
Manufacturing Employees in the US. and 
Germany: A Cross-Cultural Comparison 


Speaker: 


K. Peter Kuchinke, University of Illinois at 
Urbana-Champaign 


Topic 3: 


Technical and Occupational Education 
Instructors ’ Perceived Use of Student 
Assessment Information in Making Educational 
Decisions 


Speaker: 
Topic 4: 


Greg Belcher, Pittsburg State University 
The Education of Retail Managers Through 
Management Training Programs in Apparel 
Retail Organizations 


Speaker: 


Irene M. Foster, University of Massachusetts, 
Amherst 


Chair: 

Facilitator: 


Donna H. Redmann, Louisiana State University 
Davison Mupinga, Louisiana State University 
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3:30 - 5:00 p.m. 

Assessment in Vocational Education 



Topic 1: 

Speaker: 
Topic 2: 

Speakers: 



Assessment of Work- Based Learning Programs in 
Georgia 

Clifton Smith, University of Georgia 

Assessment in the Vocational Education Classroom: Using 

Information for Instructional Decisions 

Howard R. Gordon, Marshall University 

Richard J. Yocke, West Virginia University Institute of 

Technology 



Chair: William B. Camp, Virginia Tech 

Facilitator: Rosalyn Vaughn 
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Determining Success of Vocational Student Enrolled in Distance Education Courses 

Michael K. Swan Diane H. Jackman 

Biological Systems Engineering Department Teacher Education 

205 Smith Hall 155 EML Hall 

Washington State University North Dakota State University 

Pullman, WA 99164-6120 Fargo, ND 58105 



Introduction 

Distance education is an emerging technology intended to deliver both resident and remote site 
instruction and educators who use distance education must provide educational experiences that 
will equal resident education in terms of quality and quantity. Both resident and distance 
education are intended to provide students with valid, useful information that promotes learning. 
Resident, or host site, education occurs when the instructor and students meet at a predetermined 
location, thus providing easy face-to-face interaction. Distance, or remote site, education has 
been accepted as teaching when the instructor and students are physically separated in terms of 
location and/or time. The instructor can be in different buildings, cities, counties, states, or even 
countries. According to Swan and Brehmer (1994), distance education refers to “the 
simultaneous delivery of instruction from a host site or classroom to remote site(s), coupled with 
real time live audio and real time live video interaction between teacher and student(s) - not 
correspondence, video, or internet courses.” 

Swan (1995) noted that advancements in communications technology have dissolved some of the 
major distinguishable characteristics between distance education and traditional education. 
According to Swan and Jackman (1996), strategies of teaching at a distance and host site are 
converging because traditional teaching strategies are being abandoned or modified in favor of a 
problem-based, resource-based, or activity based approach that de-emphasizes the teacher as the 
main source of knowledge. In 1990, Moore and Thompson analyzed resident and distance 
education and developed a framework for determining the relationship between the two methods 
or strategies of instructional delivery. They noted that developing technology will eventually 
merge distance education with the traditional approach so that distinctions cannot be made 
between the two methods of strategies. However, Kelly (1990) indicated that the transition from 
resident instruction in the traditional classroom to distance education requires educators to 
develop new skills in instructional strategies, methods of teaching, timing, teacher/student 
interaction, feedback, printed supplement materials, and evaluation. 

Souder (1993) compared distance learners and traditional host site learners. The distance 
education learners performed better than the host site learners on several dimensions, including 
exams and homework assignments. This finding was attributed to the extraordinary 
commitment, higher maturity level, and motivation of the distance learner. However, this 
finding is contrary to other evidence that distance learners are at a disadvantage in their learning 
experience, especially evaluation of their cognitive performance (Moore and Thompson, 1990). 

The increasing availability of telecommunications has provided vocational or applied education 
faculty with unique opportunities to plan and deliver distance education courses and programs. 
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V ocational education students are also enrolling in more distance courses and programs due to 
availability, time, and place. However, there is a lack of studies that compare the performance of 
vocational education students receiving instruction via distance technology versus students 
receiving the same instruction through the traditional resident, host site, classroom setting. 

Purpose/Objectives 

The purpose of the study was to ascertain if student achievement differences existed in courses 
delivered via distance education. Specific research objectives were: 

1 . Describe student's enrolled, remote site and host site, in distance education courses on 
selected demographic characteristics 

2. Ascertain if differences existed between remote site and host site student achievement 
based on grades/scores (GPA) obtained by grade level. 

3. Ascertain if differences existed between remote site and host site student achievement 
(final grade received) based on individual course success. 

Methodology 



Population 

The population of remote site and host site schools was identified from an alphabetical list of 
secondary schools utilizing distance education technologies supplied by the State Department of 
Pubic Instruction. The schools were all located within one niidwestem state. Each of the 
identified schools administration were asked to participate in the study. From the total list of 
schools using distance education, a total population of schools willing to participate were 
identified, N = 46 schools. From this revised list of schools, a study sample was selected using 
appropriate cluster sampling methods outlined by Wiersma (1995). 

As each school was selected, all courses/classes being offered via distance education from that 
school were selected for this study. Each student in the study, n = 623, was enrolled in at least 
one course being offered via distance education. To retain the confidentiality of the student, 
administrators or the assigned school representative was asked to assign an identification number 
to each student. This number was used to report all data concerning that student. The researcher 
did not know or have knowledge of any student name, only their assigned number. 

Instrumentation / Data Collection 

The study instrument, adapted from the Souder study (1993), was completed from student's 
records by the administration or assigned school representative. The instrument was assessed for 
content and face validity by graduate students, teacher educators, and state supervisors in 
vocational education. Reliability of the instrument was .89 (Cronbach's alpha coefficient). They 
were asked to report gender, grade level of student, period(s) taking distance education courses, 
name(s) of specific distance education course(s), location of student (remote or host site), total 




2 



13 



daily assignment scores, exams and/or quiz scores, and final exam score. All grades reported 
were based on or converted to a 0 to 1 00 point system. If conversions were made they were 
made by the administration or assigned school representative using a scale provided by the 
researcher. This grading scale was one recommended by the state superintendents and principals 
association. 

Data Analysis 

Data were analyzed using the Statistical Package for the Social Sciences (SPSS Version 6.1) for 
Windows. Data were summarized using descriptive statistics. Frequencies, percentages, means 
and standard deviations were utilized to analyze and describe findings. One way analysis of 
variance was used to analyze differences between the grade levels of students, the location of 
student, and gender. All tests were run at the .05 alpha level. 

Results 

Objective 1: Demographic Characteristics 

Demographically, students in the study were predominately located at remote sites, 68.1% (424), 
and 31.9% at host sites (199). In the study, there were 245 male students (39.3%) and 378 
female students (60.7%). The study identified ten individual courses being offered via distance 
education, one course was eliminated from the study because no scores were made available to 

the researcher. Table 1 identifies the total number of students by grade level, 9^^ = 56 (9%), 

10*= 126 (20.2%), 11*= 161 (25.9%), and 12* = 280 (44.9%). 

Table 1 

Grade Level of all Students 



H Grade Level 



As shown in Table 2, the study group is divided into groups identified by specific course name 
and by location receiving the course. Frequencies and percentages are used to identify students 
enrolled in distance education courses all sites in the study. 

Table 2 

Individual Course Enrollment Frequencies and Percentages 



Course Name 


All Sites 




Host Site 


Remote Site 




N 


% 


N 


% 


N 


% 


Foreign Language 


231 


37.1 


91. 


39.4 


140 


60.6 


Ag Business Mgt. 


77 


12.4 


3 


3.9 


74 


96.1 


Vocational Marketing 


21 


3.4 


14 


66.7 


7 


33.3 


Natural Resources 


42 


6.8 


4 


9.5 


38 


90.5 


Math - Calculus 


119 


19.1 


63 


52.9 


56 


47.1 


Chemistry 


70 


11.2 


9 


12.9 


61 


87.1 


Art 


14 


2.2 


5 


35.7 


9 


64.3 


Statistics 


14 


2.2 


6 


42.9 


8 


57.1 


Animal Science 


35 


5.6 


4 


11.4 


31 


88.6 


Total 


623 


100 


199 


31.9 


424 


68.1 



Objective 2 



One way analysis of variance was used to test if differences in student achievement existed 
between remote and host site students based on mean GPA. No significant differences were 
found. 

Table 3 identifies the mean Grade Point Average (GPA) of students located at remote sites and at 
host sites. The grade point averages of students enrolled in distance education courses at both 
the remote and host sites were very similar. 

Table 3 



Grade Point Average According to Location Receiving Course 



Location 


N 


GPA 


SD 


Remote Site 


424 


3.19 


.76 


Host Site 


199 


3.14 


.84 


Total All Sites 


623 


3.18 


.78 




“ 15 



ANOVA was used to test if differences in student achievement existed between grade levels 

tVi tVi 

based on mean GPA. There were significant differences among the four groups (9 grade, 10^“ 
th th 

grade, 1 1 grade, and 12 grade). The analysis of the data yielded an F value of 2.84 with a p 

of .037 as reported in Table 4. Ninth grade students earned a GPA significantly higher than 1 1^^ 

th t"h 

grade students and 9 grade students earned a significantly high GPA than did 12^“ grade 

students as reported in Table 5. 

Table 4 

Analysis of Variance for Grade Point Average on Grade Level 

Source df SS MS F p 

Between 3 5.20 1.73 2.84 .037 

Within 619 377.32 .61 



Table 5 

Grade Point Average of students bv grade level and bv site 

Grade Level / N GPA SD 

Site 



9^^ Host Site 


2 


3.35 


.92 


9^^ Remote Site 


54 


3.43 


.73 


9^^ Total 


56 


3.43 


.72 


10^^ Host Site 


70 


3.19 


.71 


10 Remote Site 


56 


3.25 


.58 


10* Total 


126 


3.22 


.65 


11 * Host Site 


71 


3.31 


.57 


1 1 * Remote Site 


90 


3.08 


.79 


11 * Total 


161 


3.18 


.71 


12* Host Site 


56 


2.87 


1.16 


12 * Remote Site 


224 


3.16 


.78 


12* Total 


280 


3.10 


.87 
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Objective 3 



One way analysis of variance was used to test if differences existed between remote site students 
GPA and host site students GPA by individual course. There were no significant differences 
among the two groups (remote site and host site). The analysis yielded an F value of .51 with a p 
= .47 as reported in Table 6. There were significant differences between the groups by grade 
level. The analysis yielded an F value of 12.23 with a p = <.0001 . Analysis of data of student 
achievement (GPA) by remote site or host site by individual course identified no significant 
differences. The analysis yielded an F value of .77 with a p = .62 as reported in Table 6. 

Table 6 



Analysis of Variance for Grade Point Average on Site Location and Individual Course 



Source 


df 


SS 


MS 


F 


D 


Site 


1 


.262 


.262 


.511 


.4749 


Class 


8 


50.194 


6.274 


12.231 


<.0001 


Site * Class 


8 


3.186 


.398 


.776 


.6237 



When comparing student GPA by course we see differences in individual courses. All courses 
GPA's were identified as being significantly higher than GPA's in Vocational Marketing. 
Foreign Languages GPA's were significantly higher then GPA's in Vocational Marketing. 
Natural Resources GPA's, Chemistry GPA's, and Art GPA's were significantly higher than 
GPA's in Foreign Languages. Natural Resources GPA's were significantly higher than GPA's in 
Ag Business Mgt., Math, and Animal Science as reported in Table 7. Significant differences 
were found when grouping traditional vocational courses, Ag Business Mgt., Vocational 
Marketing, Natural Resources, and Animal Science, together and comparing to the traditional 
academic in student achievement as measured by Grade Point Average. Students in traditional 
academic courses (3.25 GPA) had a higher GPA than did students in vocational courses (2.99 
GPA). The analysis yielded an F value of 13.56 with a p = .0003 as reported in Table 8. 

Table 7 



Individual Course Success CGPA) by Host Site and Remote Site 

Course Name Host Site Remote Site 

N GPA N GPA 



Foreign Language 


91 


3.29 


140 


3.06 


Ag Business Mgt. 


3 


3.23 


74 


3.09 


Vocational Marketing 


14 


1.70 


7 


2.00 


Natural Resources 


4 


3.68 


38 


3.43 


Math - Calculus 


63 


3.11 


56 


3.21 


Chemistry 


9 


3.57 


61 


3.58 


Art 


5 


4.00 


9 


4.00 
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Statistics 
Animal Science 



6 

4 



3.50 

3.43 



8 

31 



3.50 

2.88 



Table 8 

Analysis of Variance for Grade Point Average on Academic and Vocational Courses 

Source df SS MS F 2 

Between 1 8.18 8.18 13.56 .0003 

Within 621 374.34 .60 



• Conclusions, Implications and/or Recommendations 

Students enrolled in distance education courses were primarily located at remote sites. Remote 
site students were provided the opportunity to take these courses because courses were being 
offered via distance education technology. Without this opportunity, most of these students 
(424) would not have been able to enroll in these courses. As students gained experience with 
distance education and grade level they enrolled in more courses being offered via distance 
education. 

Receiving instruction by distance education resulted in no differences in grade point average for 

all students at either remote site or host site. Students in 9 grade did have higher grade point 
th th 

averages then did 1 1 grade and 12 students. This can be attributed to the specific courses 
and complexity of courses taken by these groups of students. Twelfth grade students located at 

host sites had a significantly lower GPA than 12 grade students located at remote sites. This 
area needs further analysis to determine exact reason for this occurrence. 

Significant differences did not exist between all remote site students and all host site students in 
this study. It did make a difference if a student was in the same room with the teacher or if they 
were at a different location. Significant differences were found when grouping students by grade 

tVi tVi 

level and if they were located at remote site or host site. Remote site students in 9 ,10 and 

th 

12 grades had a higher GPA than did their counterparts located at the host site. 

Individual courses being offered via distance education revealed differences in student 
achievement. Vocational Marketing was significantly different than all other courses. After 
placing a phone call to the local administrator it was determined that the cause could have been 
that the teacher was in their first year of teaching and this was the first distance education course 
they had taught. Generally it did not matter if the student was at the remote site or host site for 
the instruction, student success was high or above average on all except one course. 

Grouping traditional academic courses and comparing to the vocational courses offerings 
significant differences were revealed. Students taking traditional academic courses were 
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receiving a higher GPA than those students taking vocational courses. This could have been 
because of the lack of applied hands-on activities in the vocational courses. 

The results of the study offer promise that distance education courses do provide opportunities 
for students who would otherwise not be able to take these courses. Additionally student 
success, GPA, was above average (mean GPA whole group = 3.18) in these courses. 

Further research needs to be conducted with populations of students to determine if there are 
differences in learning styles of students who are enrolled in distance education courses. This 
could have attributed to the success rate of students enrolled in these courses. Additionally, the 
quantity of distance education courses being taken by students may be a contributing factor to 
success. 
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The Use of Applied Communications in 
Developing Critical Thinking Skills of Tech Prep Students 

Don R. Gelven, Dean 
Linn State Technical College 
Linn, Missouri 

Introduction and Theoretical Base 



Bob R. Stewart, Chair 
Council for Vocational Education 
University of Missouri 
Columbia, Missouri 



Employers and educators are generally in agreement that students need to increase their 
problem solving and critical thinking skills. Numerous national reports have pointed out that 
students are deficient in this area. The Tech Prep initiative has tended to focus on activities 
designed to better prepare students for the world of work (Custer, Ruhland, & Stewart, 1997). 

Various approaches have been proposed for improving the problem solving and critical 
thinking skills of students. Three approaches found in the literature were the general, infusion, 
and immersion methods which Sormunen (1992) discussed. The general method of teaching 
critical thinking is by a separate course outside of any content area. The infusion method 
consists of teaching general principles of reasoning within the context of subject matter. The 
immersion method focuses on the knowledge content without pointing out general critical 
thinking principles. 

The theoretical base of this study was formed from cognitive and behavioral learning 
theories and theories of self-efficacy as applied to problem solving and critical thinking. These 
learning theories can be categorized as theories that fall within two broad areas. The first area 
deals with the nature of problem solving and critical thinking. The second with whether problem 
solving and critical thinking can be taught, and how to teach it if it can be taught. 

The ability to think critically and to solve problems has been a concern of philosophers, 
educators, and psychologists for many centuries. Sternberg (1986) stated that Plato and Aristotle 
founded the critical thinking movement. These philosophers were concerned with the first area 
of theory. Sternberg attributes the modern-day movement to John Dewey. Dewey (1933) was 
concerned with the nature and value of thinking. He considered thinking to be the process by 
which individuals find meaning in the world in which they live. The ability to think critically is 
a prerequisite for problem solving and as such is of significant value. Dewey believed that the 
ability to think critically, or reflectively, was a function of one's experience as well as one's 
intellect. 

The basis for the second area of learning theory, that which deals with methods for 
developing cognitive abilities, can be traced to Bloom (1956). Bloom developed a taxonomy of 
cognitive levels of learning. Each level requires a different mental activity or way of thinking. 
Lower levels of learning are a prerequisite for higher levels, where higher levels are often 
referred to as requiring higher-order thinking skills. Problem solving requires higher-order 
thinking, which according to Bloom can be taught. 

A number of authors have proposed methods for teaching thinking and problem solving. 
For example, Bruner, Goodnow, & Austin, (1956), Gallagher (1993), Halpem (1984), and 
Ruggiero (1988) have described various aspects of thinking, learning, and problem solving and 
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methods for developing problem solving skills. The methods they proposed are based upon both 
cognitive and behavioral learning theories. 

The basis for theories of self-efficacy can be attributed to Bandura (Greenberg and Baron, 
1993). They discussed the development of self-efficacy. The theory is that self-efficacy 
develops "partly through direct experiences, in which individuals perform various tasks and 
receive feedback on their success, and partly through vicarious experiences, in which they 
observe others performing various tasks and attaining varying levels of success at them" (p. 208). 
They further stated that, regardless of how it develops, “the stronger individuals’ beliefs that they 
can perform successfully, the higher their performance actually tends to be” (p. 208). 

Problem solving is one of the foundation skills needed for high-skill, high-wage 
employment (United States Department of Labor, 1992; Daggett, 1992). Life in technologically 
oriented countries, such as the United States, is characterized by rapid change. This rapid change 
complicates life and makes it necessary for students to learn how to be effective problem solvers. 
It is particularly desirable that students learn to be effective problem solvers in the context of 
actual production in work situations. In order to keep up with international competition and the 
faster pace of technological change, employers in the United States are demanding that the 
public schools be responsible for developing the critical thinking and problem solving skills of 
students. In response to this demand, goals have been defined at both the national and state level 
for developing students’ problem solving skills. 

The fifth objective of National Education Goal 6 calls for increasing the proportion of 
college graduates who demonstrate the ability to think critically and solve problems. 

Specifically, the objective is that: “The proportion of college graduates who demonstrate an 
advanced ability to think critically, communicate effectively, and solve problems will increase 
substantially” (National Education Goals Panel, 1994, p. 10). 

This study investigated the extent to which the development of problem solving self- 
efficacy and critical thinking skills, based upon learning theory and theory explaining the 
development of self-efficacy, were accomplished in a Tech Prep applied communications course. 

Purpose 

The primary purpose of this study was to examine the effect of participation in a Tech 
Prep program and specifically an applied communications course on the problem solving self- 
appraisal and selected aspects of critical thinking skills of secondary students. A secondary 
purpose was to examine the effects of a traditional English course and an honors English course 
on these factors. The study was conducted to answer the following questions: 

1. Is there a significant difference in the mean total scores from a self-appraisal of problem 
solving abilities and from a measurement of critical thinking skills among students 
completing an applied communications course, a traditional English course, or an honors 
English course? 

2. Is there a significant difference in students’ self-appraisal of their problem solving abilities 
and in selected aspects of students’ critical thinking skills when measured near the beginning 
and near the end of an applied communications course, a traditional English course, or an 
honors English course? 
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3. Is there a significant relationship between the students’ self-appraisal of their problem 
solving abilities and their critical thinking skills for students completing an applied 
communications course, a traditional English course, or an honors English course? 

Methods 



A pretest-posttest nonequivalent control group design was used for the study. 
Demographic information and two instruments were used to collect data. 

The instrument used to measure the individual’s self appraisal of their problem solving 
self-efficacy for this study was the Problem Solving Inventory (PSI) Form B (Heppner, 1988). 
The PSI contains 35 items and uses a 1 to 6 Likert-type scale with 1 representing strong 
agreement and 6 strong disagreement with the statement. Reliability for the PSI was reported to 
be r =.89 for test-retest reliability and r =.90 for internal consistency. The instrument used to 
measure selected aspects of the students’ critical thinking abilities was the Watson-Glaser 
Critical Thinking Appraisal (WGCTA) Form B (Watson and Glaser, 1980). The instrument 
consists of 80 items divided into five subscales of 16 items each and includes exercises which are 
purported to be examples of problems, statements, and interpretations of data which are regularly 
encountered at work or school. The split-half reliability estimates for eleventh grade students 
was reported to be r =.79. The maximum raw score for the WGCTA is 80. 

The purposive sample consisted of the students enrolled in four sections of each of three 
different English courses. The three courses were comprised of an applied communications 
course, a traditional English course, and an honors English course. Where there were more than 
four sections of a course, the sections for this study were randomly selected. The total number of 
students enrolled in the 12 sections included in this study was 254 at the time of the pretest, 
increasing to 279 by the time of the posttest. Complete data, which consisted of pretest and 
posttest measurements for both the PSI and WGCTA, were obtained from 136 students. These 
136 students included 53 Honors English III students, 43 English III students, and 40 applied 
communications students. All of these students were in the 1 1th grade, with the exception of one 
10th grade student who was assigned to English III, and three 12th grade students who were 
assigned to applied com m unications. 

The Statistical Analysis System (SAS) was used to calculate a MANOVA (multivariate 
analysis of variance ) with the general linear model adaptation for a two factor repeated measures 
experiment or a Pearson Product-Moment correlation, as appropriate. The null hypotheses were 
rejected if the F value was significant at equal to or less than the .05 alpha level. 

Findings and Conclusions 

The mean pretest and posttest scores (Table 1) on the PSI for the Honors English students 
was 83.7 and 79.0; for the English III students was 92.4 and 90.8; and for the applied 
communications students was 93.2 and 95.7. The mean pretest and posttest scores on the 
WGCTA for the Honors English students was 57.5 and 59.1; for the English III students was 
48.5 and 45.6; and for the applied communication students was 43.1 and 42.6. 



Table 1 

Mean Pretest and Posttest Scores for the PSI and WGCTA 



Variable 


M 


n 


SD 


Variance 


SE 


Course 1 -Honors English III 
Pretest PSI 


83.736 


53 


17.811 


317.237 


2.447 


Posttest PSI 


79.000 


53 


19.104 


364.962 


2.624 


Pretest WGCTA 


57.453 


53 


6.256 


39.137 


.859 


Posttest WGCTA 


59.151 


53 


7.140 


50.977 


.981 


Course 2-Enelish III 
Pretest PSI 


92.419 


43 


14.945 


223.344 


2.279 


Posttest PSI 


90.791 


43 


18.809 


353.788 


2.868 


Pretest WGCTA 


48.488 


43 


7.830 


61.303 


1.194 


Posttest WGCTA 


45.628 


43 


8.449 


71.382 


1.288 


Course 3 -English III C 
Pretest PSI 


93.225 


40 


19.453 


378.435 


3.076 


Posttest PSI 


95.725 


40 


21.664 


469.333 


3.425 


Pretest WGCTA 


43.100 


40 


7.063 


49.887 


1.117 


Posttest WGCTA 


42.575 


40 


6.898 


47.584 


1.091 



The first hypothesis, HOI, stated there is no statistically significant difference in the 
mean total scores from the PSI and WGCTA by type of English course assignment. HOI was 
rejected at the .05 level of significance. As reported in Tables 2 and 3, there was a significant 
difference in mean total scores of students by English course assignment. Students assigned to 
the honors English III course had mean total scores which were significantly more positive than 
the mean total scores of students assigned to English III. The mean total scores of English III 
students were, in turn, significantly more positive than the mean total score of applied 
communication students. 

Table 2 

Pillai's Trace Test for Multivariate Analysis 



Value 


Source 


F 


NDF 


DDF 


PR>F 


Course 


.78878902 


43.3075 


4 


266 


.0001* 


Time 


.01707687 


1.1467 


2 


132 


.3208 


Interaction 
♦ 


.09904930 


3.4650 


4 


266 


.0088* 



* Significant 
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Table 3 

ANOVA for PSI and WGCTA 



PSI 


DF 


SS 


MS 


F 


PR>F 


Course 


2 


9076.396 


4538.198 


7.82 


.0006* 


Error 


133 


77163.659 


580.177 






Time 


I 


1 1 1.202 


1 1 1.202 


.95 


.3322 


Interaction 


2 


596.763 


298.381 


2.54 


.0825 


Error 


133 


I56I3.I74 


1 17.392 






R-square=.847 












WGCTA 


DF 


SS 


MS . 


F 


PR>F 


Course 


2 


I2I43.342 


607I.67I 


75.67 


.0001* 


Error 


133 


I067I.936 


80.240 






Time 


I 


21.208 


21.208 


.83 


.3632 


Interaction 


2 


247.125 


123.562 


4.85 


.0093* 


Error 


133 


3388.153 


25.474 






R-square=.87I 













* Significant 



The second null hypothesis, H02, stated there is no statistically significant 
difference in the mean total scores from the PSI and WGCTA by time of administration of 
the test instruments. H02 was not rejected at the .05 level of significance. As reported in 
Table 2, there was no significant difference between the pretest and posttest mean total 
scores from the PSI and WGCTA, for any class of students. 

In addition, results were examined to check for interactions. An analysis of 
variance procedure revealed that there was a significant interaction between the mean 
WGCTA scores by the type of English course assignment and time. A post hoc test (Table 
4) was used to isolate source of the differences between the interaction of course and time. 
It was ascertained that the mean pretest and posttest WGCTA test scores of honors English 
III students were significantly higher than the mean pretest and posttest WGCTA scores of 
English III and applied communications students. Also, English III student’s pretest and 
posttest scores were significantly higher than the pretest and posttest scores of applied 
communications students. 




14 



25 



Table 4 

Least Square Mean. Least Square Standard Error and Least Sipnifir.ant 
Difference Test for Dependent Variables PSI and WGCTA bv Course 



Course 


n 


M 


SE 




LSD Test 
2 3 


Denendent Variable=PSI 
Honors English III 


106 


81.367 


1.052 


1 


.0001* 


.0001* 


English III 


86 


91.604 


1.168 


2 




.0904 


English me 


80 


94.475 


1.211 


3 






Dependent Variable=WGCTA 
Honors English III 


106 


58.301 


.490 


1 


.0001*" 


.0001* 


English III 


86 


47.058 


.544 


2 




.0001* 


English me 

♦ e: — 


80 


42.837 


.564 


3 







* Significant 



The third null hypothesis, H03, stated the correlation between the student’s scores 
from the PSI and from the WGCTA is not statistically different from zero. H03 was tested 
using the Pearson Product-Moment calculation. It was ascertained that the correlation of 
.41 for the students’ posttest scores from the PSI and WGCTA was significantly different 
than zero. Therefore, null hypothesis H03 was rejected. 

The results of this study do not support a conclusion that the critical thinking skills 
of any of the three groups of subjects of this study were significantly changed during the 
course of the school year. Therefore, it cannot be concluded that the applied 
communications course significantly changed students’ critical thinking skills. On the 
other hand, it cannot be concluded that the honors English III or the traditional English III 
courses significantly changed students’ critical thinking skills. 

Implications 

These results may support a belief that critical thinking skills and problem solving 
skills caimot be developed in a short period of time. This explanation is consistent with a 
conclusion reached by Langholz and Smaldino (1989). They pointed out that there is not 
much evidence to support the conclusion that critical thinking and problem solving can be 
developed in a short period of time. 

There is evidence from the literature that would support the idea that training can . 
develop a more positive self-appraisal of an individual’s problem solving abilities. 
Gallagher (1993) pointed out that the PSI has been used as an outcome measure for 
problem solving seminars. In a study of an eclectic approach to training paraprofessionals 
in counseling, Gallagher administered the PSI both pre and posttest and found that the PSI 
scores of trainees decreased significantly, indicating that the trainees’ self-appraisal of their 




15 26 



problem solving was more positive after training. However, the self-appraisal of their 
problem solving abilities by the subjects included in the present study did not significantly 
change over the time of the study. In addition, no statistically significant evidence was 
found indicating that the spread between the self-appraisal of Honors English III students 
as compared to the self-appraisal of applied communications students increased over time. 

Although the change was not statistically significant, the posttest mean PSI scores 
of Honors English III and English III students were lower than -their pretest mean scores. 
This could indicate that a slightly more positive self-appraisal of their problem solving 
abilities was developed. By way of contrast, the posttest mean PSI score (95.73) of the 
applied communications students was higher than their pretest mean score (93.23). A 
possible explanation for this slightly less positive self-appraisal might be that they 
developed a more realistic awareness of their problem solving abilities during the applied 
communications course. 

Student’s grade point averages and PL.AN test scores were also examined to 
provide additional background information for the study. An examination of the data 
revealed that both the grade point average (3.5 vs. 2.2) and PLAN test scores (23.1 vs. 

14.2) of the honors English students were higher than for applied communication students. 
Students’ scores from the PLAN test, which requires higher order thinking skills, were 
significantly related to their PSI and WGCTA scores. This indicates that the students 
assigned to the different courses had different characteristics which likely influenced their 
performance on the test instruments. 

The results of this study indicate that there was no significant improvement in the 
critical thinking abilities of students who complete the applied communications course. It 
further indicates that these students’ perceptions of their problem solving skills does not 
become more positive as a result of completing this course. 

The Tech Prep curriculum for the applied communications course should 
incorporate specific objectives for improving the critical thinking skills of students and for 
developing a more positive perception of their ability to solve unstructured problems of the 
type encountered at home and at work. To achieve these objectives, instruction 
specifically designed to develop inductive and deductive reasoning skills and to develop 
the ability to draw inferences should be incorporated into the Tech Prep applied 
communications curriculum. 

In addition, other activities and courses associated with the Tech Prep initiative 
should be examined to ascertain to what extent they contribute to the development of 
critical thinking skills of students. Such information is needed as we face the challenge of 
better preparing all students to participate in a global and technically oriented work force. 
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USING THE DELPHI TECHNIQUE TO CREATE 
A DEFINITION OF INTERNATIONALIZATION 



Barbara G. Ludwig, Associate Professor, Ohio State University Extension 
2120 Fyffe Road, Columbus, Ohio 43210. E-mail; ludwig.2@osu.edu 

In the 21st century, an educated person will need to 
function effectively and responsibly in a global 
environment both in day-to-day work and in some social 
_ and civic interactions. A global perspective is requisite 
to good citizenship, and must be a focal point of all our 
educational and research programs. 

(Sammons, 1998, p.l) 

INTRODUCTION and THEORETICAL BASE 

What characterizes an internationalized Extension system in the U.S.? A simple question, but 
one which few individuals have sought to address tlu-ough research. Henson (1990) in his study 
of university internationalization stated that internationalization was frequently viewed in 
general, rather amorphous terms that were difficult for some to understand and comprehend. 
Arum and Van de Water (1992) in the book Bridges to the Future: Strategies for 
Internationalizing Higher Education supported this view. In article after article, and at 
conference after conference the terms used to characterize the international dimension of 
education vary tremendously. A review of literature revealed broad, but often ambiguous goal 
statements related to internationalization of Extension (King & Martin, 1991; Rosson & Sanders, 
1991; Poston & O’Rourke, 1991; Andrews & Lambur, 1986). Some ideas had been formulated 
for higher education, but for Extension none defined internationalizing in terms of objectively 
verifiable indicators of success. 

Identifying characteristics of an internationalized Extension system can help organizations focus 
available resources on creating changes needed to achieve internationalization. This becomes 
increasingly important with the release of a task force report from the International Agriculture 
Section of the National Association of State Universities and Land-Grant Colleges. Globalizing 
Agricultural Science and Education Programs for America defines an implementation plan to 
help position U.S. agriculture for the 21st century. It creates a vision of “globally competent 
stakeholders, faculty and students... who live, compete, and work well in an every dynamic and 
interdependent world community” (GASEPA, 1998, p. 6). Extension and outreach education are 
an integral part of the plan. 

Kaufman (1982,1992) suggested putting problems into the context of what is and could be when 
dealing with organizations. The Organizational Elements Model provided a framework for the 
study. Kaufman’s model used a holistic framework in looking at organizations and what those 
organizations use, do and deliver. 
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STATEMENT OF THE PROBLEM: 



The purpose of the study was to identify the characteristics that will describe an 
internationalized state university Extension system. 

METHODOLOGY: 

The study used a three-round, modified Delphi technique to explore and describe the 
characteristics of an internationalized state university Extension system. Delphi, a group 
process, utilized individual written responses to three researcher developed instruments. The 
process was further characterized by multiple iterations designed to accomplish conyergence of 
opinion. The methodology was determined by the nature of the problem to be investigated and 
the time and cost restraints of bringing together the individuals needed to address the problem. 
Participants’ anonymity was maintained during the three rounds of the study. 

Panel Selection 

Delphi operates on the principle that several heads are better than one in making subjective 
judgements about the future and that experts will conjecture based on rational judgements rather 
than merely guessing. The Delphi Panel members were purposefully selected following a 
nomination process. A total of 15 individuals, well known and respected for their contributions 
to Extension, land-grant colleges or universities in the area of internationalization were 
identified. 

The participants selected by the review panel met at least three of the criteria established for 
selection. The criteria were: (1) national/intemational reputation; (2) familiarity with the topic; 
(3) conducted research, written or lectured on the topic; (4) considered to have a deep interest in 
the problem and important knowledge or experience to share. The Panel identified the degree to 
which they believed each item on the instrument contributed to the internationalization of a state 
university Extension system 

Instrumentation 

The initial instrument contained 39 position statements derived from the literature and structured 
interviews with international experts. A seven-point Likert-type scale was used with 0 indicating 
“no importance” and 6 indicating “critical importance.” 

Use of a content validity panel assured face and content validity of the initial instrument. Given 
the nature of the Delphi technique, additional types of validity and reliability estimates were not 
appropriate for the instrument. Reliability, a benchmark criterion for assessing the consistency 
of the instrument in measuring accurately whatever it sets out to measure, cannot be determined 
by conventional means in a Delphi study. Reliability-estimation procedures look at stability in 
measurement over time or across forms. Changing responses from one round to the next is 
encouraged as the Delphi Panel moves toward consensus. The instrument is modified in each 
round based on input from panel members. Reproducibility using the same approach with 
different experts could be considered to play somewhat the same role as reliability. Dalkey 
(1969) reported an increase in reliability of group responses with increasing group size. 
Reliability with a correlation coefficient approaching .9 was found with a group size of 13. 




The Delphi Panel was asked to identify the degree to which they believed each item on the 
instrument contributed to the internationalization of a state university Extension system. A 
seven-point Likert-type scale was used. Delphi Panel members were asked to support their 
opinion with a rationale. Space was also provided for panel members to add new statements. 
Delphi Panel responses were incorporated in successive instruments. A total of fifty-one items 
were considered during the three rounds of the Delphi 

Presentation of feedback is central to the operation of the Delphi. Feedback to participants has a 
tendency to move the Panel toward consensus. The study used a combination of statistical and 
verbal feedback. For Rounds II and III, a statistical summarizing for each item in the form of a 
frequency table and an anonymous listing of all comments made by panel members encouraged 
reassessment of initial judgements. 

Data Collection 

The Delphi instruments were mailed to the Panel using U.S. mail or air mail to international 
locations. A variety of techniques were used to ensure maintenance of interest and participation 
in the study. 

Data Analysis 

The computer program SPSS was used for data analysis. For each round, items on which 
consensus was reached were identified. Consensus on an item was considered to have been 
reached when 80% of the ratings fell within two categories on a seven-point scale. Frequency 
coimts and percentages, along with the mode and median were reviewed in determining 
consensus. For each round, those items not meeting the criteria for consensus were included in 
the following roimd as well as new items generated from suggestions. 

Following round III, statistics of central tendency and variability were calculated for all items on 
which consensus had been reached. The mean was used to describe the level of importance of 
the item to an internationalized state Extension system. 

FINDINGS 

The results of the study represent the collective opinion of the experts participating in the Delphi 
Panel at a single point in time and cannot be construed to be representative of any other 
population or situation. Fourteen of the 15 participants responded to each round, a 93% response 
rate. Fifty-one items were considered. Consensus was achieved on 38 items identified as having 
moderately high importance to critical importance for the internationalization of a state 
university Extension system. Table 1 reports the items where consensus was reached. 

Consensus was not achieved on thirteen items after three rounds. Comments made by the Panel 
during each round and reported anonymously provided additional information to describe the 
ratings and clarify issues. Three hundred and sixteen comments were received. 

Following Kaufman’s model (1982,1992), the results were categorized as organizational efforts 
and organizational results. Reported below are Efforts and Results that were identified as 
having critical or high importance. 



Organizational Efforts include existing conditions that affect organizational activities as well as 
methods and procedures necessary for managing these activities. An effort of critical 
importance to the internationalization of Extension was recognition by Extension professionals 
of the relationship between international issues and the Extension mission. A commitment to 
international education as a part of the Extension mission, an organizational culture that expects 
international activity and administrators who communicate support for internationalization were 
viewed as highly important. International issues include: knowledge of international 
agriculture, commitment to human development, the debate on privatization of Extension and the 
experiences of Extension systems in seeking new ways of funding services. Financial support 
for internationalization activities and a person identified to provide leadership to 
internationalizing efforts were highly important. 

A critically important effort was development of a personnel evaluation system which 
recognized international efforts. Highly important was having a reward structure that recognizes 
internationalization in its system of rewards. These rewards include merit adjustments, tenure, 
promotion and peer recognition. Staff development processes identified as having high ~ 
importance included: interaction with scholars from other countries to increase faculty expertise, 
opportunity for international experiences for county agents and administrators and professional 
improvement activities designed to increase knowledge of global issues. 

Organizational Results, according to Kaufman, include the products or programs/outputs 
developed and their delivery to stakeholders through educational programming efforts. The 
Result having critical importance to internationalizing Extension was clientele developing a 
fundamental understanding of global and national interdependence. 

Other Results identified as having critical importance included: Educational programs offered 
in the United States that stress the impact of international economics forces on agricultural 
markets and Extension educators incorporating international perspectives into ongoing 
educational activities. Programs that help clientele understand complex worldwide issues and 
programs that train local business persons for participation in international markets were 
identified as having high importance. These programs might include having key leaders 
participate in interdisciplinary international experience. Highly important for Extension was 
targeting commodity groups for public policy education on global decision making and rural 
clientele for education on the international marketplace. Also, of high importance were 
Extension educators assisting communities in building a sense of responsibility for the wise use 
of natural resources in the context of global trends. 

CONCLUSIONS 

The study brought greater clarity and focus to the characteristics of an internationalized 
Extension system. Internationalization was not seen as a fourth dimension to the work of faculty 
in a land grant institution: teaching, research, service and international efforts. Instead, 
successful internationalization was identified as integrating global perspectives into the basic 
mission and mandate of Extension and each of the functions of teaching, research and service. 
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Five critical elements were identified by the Delphi Panel as present in an internationalized state 
university Extension system. The absence of any one of these critical elements would indicate 
that the Extension system could not be considered internationalized. These elements were; 

♦Clientele develop a fundamental understanding of global and national interdependence 
♦ Extension educational programs within the U.S. stress the impact of international 
economic forces on agricultural markets 

♦Extension educators incorporate international perspectives into ongoing activities 
♦Extension faculty/agents recognize the relationship between basic international issues and 
the Extension mission 

♦Personnel evaluation systems recognize international efforts 

From these critical elements a definition of internationalization was developed. It states: 

Internationalization of Extension is the incorporation of international dimensions into Extension 
teaching, research and service assisting stakeholders to develop an understanding of global 
interdependence and competition. 

IMPLICATIONS FOR PRACTICE 

Eighty percent of Extension directors indicated their state either had achieved a low level or had 
not achieved any level of globalization in a study by Poston and O’Rourke (1991). For these 
Extension systems, internationalization will represent a significant organizational change. 
Identification of characteristics essential to an internationalized Extension system can help 
Extension leaders and university administrators to identify and focus available resources where 
the greatest impact can be realized. The results of the study are consistent with recent GASEPA 
(Globalizing Agricultural Science and Education Programs for America) recommendations from 
NASULGC and the International Committee on Organization and Policy. GASEPA (1998) 
identified five goals for incorporating an international dimension into teaching, research and 
extension programs. . These include: (1) enhancing global competitiveness of U.S. Agriculture 
through human resource development; (2) development and dissemination of information about 
markets, trade and business opportunities; establishment of mutually beneficial collaborative 
global partnerships; (4) promotion of trade through global economic development; (5) promotion 
of global environmental quality and stewardship of natural resources management. 

Discussed below are implications for: policy and resource decisions, targeting of stakeholders 
and staff development. 

Policy and Resource Decisions 

Extension leaders who provide a clear and enthusiastic sense of direction for internationalization 
will help to ensure concerted and sustained action. Appointing a person to support and 
coordinate internationalization of the Extension program and activities is a necessary 
implementation strategy as is incorporation of fiscal support into the ongoing Extension budget. 
The Delphi Panel did not view international experiences as critical characteristics of an 
internationalized Extension system, but high importance was placed on the organization 
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providing opportunities for faculty to work and pursue international assignments. Policy 
mechanisms that support travel abroad, professional leaves, international assignments or 
time/resources to develop curriculum should be considered. Incorporation of international 
expectations into position guidelines, reward of middle managers for fostering positive change 
toward internationalization and hiring candidates with international experience will enhance the 
human resource base of Extension. Finally, tied to the motivation of Extension educators to 
internationalize themselves and their program is recognition through the organization’s personnel 
evaluation and reward system. The Panel viewed peer recognition as an integral element of any 
reward system developed. Extension leaders need to work with promotion and tenure 
committees to define international expectations. 

Targeting Stakeholders 

Extension educators have the responsibility to help clientele develop an understanding of global 
issues. Issues that should be targeted include: human health and wellness, environmental quality 
and stewardship, diversity, development of a safe and available world food supply, renewable 
resources, the agricultural market and development of leadership skills needed in a global 
society. 

Extension has a unique role to play in helping traditional rural and agricultural clientele 
recognize the need for education on international issues and the relationship of trade and global 
economic development. Extension has typically tried to be responsive to local needs, but here a 
need exists that may be invisible to the general population and Extension needs to provide the 
leadership. As Extension expands to embrace the world. Extension must do it with local 
constituents, not for them. John Burman (1998), Vice President of Cargill, emphasized that 
understanding the importance of global markets is crucial for agribusiriesses to remain 
competitive and rural communities to remain viable. Because of Extension’s close tie with the 
agricultural sector, development and dissemination of information about markets, trade and 
business opportunities should be encouraged. Commodity groups could be targeted for public 
policy education on global decision making. Key leaders will benefit from interdisciplinary 
international experiences aimed at establishing mutually beneficial global partnerships and 
creating a greater awareness and understanding of global environmental concerns. 

Staff Development 

Extension educators need excellent technical skills, strong communication and people skills as 
they work with an audience who becomes more diverse each year. Global experience and 
attitudes increase in importance as universities target helping stakeholders to become globally 
competent. The Delphi Panel emphasized incorporation of international dimensions into 
domestic programs. One panel member explained: “one of the major problems with the 
successful internationalization of Cooperative Extension is the lack of recognition that 
international content and activities are an integral part of what clientele need. Instead the 
development assistance mentality prevails which continues to identify within Extension that 
international is something separate and different from what faculty are supposed to do. Faculty 
need to know and learn about the potentials of international programs and activities to enhance 
the quality, relevance and impact of their programs and responsibilities.” 
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Extension educators must develop global knowledge and skills that can be transmitted to 
clientele through programming efforts. For some individuals, motivation and expertise 
development will come because of out-of-county opportunities. Most Extension faculty will not 
travel beyond U.S. borders so other methods must be identified. Staff development offerings 
could include: workshops, mentoring, developing of a resource library, hosting of international 
guests, or participation in self-directed learning. Development Extension educators’ abilities to 
incorporate international perspectives into ongoing programs requires more than a brief study 
tour to another country or a single workshop. Having a small grants program to foster individual 
development or the infusion of international modules into existing programming is suggested. 
Policy mechanisms that support travel abroad, professional leaves, participation in Fulbright 
programs and international assignments will further enhance efforts to internationalize the 
curriculum. Creating linkages with corporate America to strengthen partnerships and with 
international universities should be investigated. 

FURTHER STUDY 

One outcome of the current study was the generation of additional questions and avenues for 
research. Replication of the current study is suggested. Substituting the more current term of 
globalization for internationalization should be considered. Identifying characteristics of state 
Extension systems that by reputation are internationalized and comparison to the critical 
elements identified in the study should be undertaken. Identifying factors which stimulate an 
uninvolved Extension system to begin the process of internationalization could be studied. 
Researchers could also identify barriers to Extension involvement in internationalizing and 
approaches taken to overcome barriers. 
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Table I 

Characteristics Having Importance to Extension Internationalization 

/ 



Item Mean 


SD 


Category . 


Clientele develop a fundamental understanding of global 5.85 


.38 


R 


and national interdependence. 






Extension educational programs within in the U.S. stress the 5.69 
impact of international economic forces on agricultural markets. 


.86 


R 


Extension educators incorporate international perspectives 5.54 

into on-going educational activities. 


.66 


R 


Extension faculty/agents recognize the relationships between 5.54 
basic international issues (e.g. knowledge of international 


.66 


E 


agriculture, commitment to human development, significance 
of privatization)and the Extension mission. 






Personnel evaluation systems recognize international efforts. 5.50 


.76 


E 


Key leaders participate in interdisciplinary international 5.36 

experiences. 


.74 


R 


Sensitivity to diversity issues by Extension clientele is 5.36 

enhanced. 


.63 


R 


Reward structure recognizes internationalization in its system 5.31 
of rewards. These include merit adjustments, tenure, promotion, 
and peer recognition. 


.63 


E 


Financial support for internationalizing activities is available. 5.21 


.43 


E 


Administrators clearly communicate support for 5.14 

internationalization. 


.66 


E 


A person(s) is identified to provide leadership to 5.14 

internationalizing efforts. 


.53 


E 


International experiences are provided for county 5.08 

agents who do not have faculty status. 


.64 


E 


Policy and operating procedures facilitate international 5.07 

program efforts. 


.62 


E 


The organization culture expects international activity. 5.07 


.62 


E 


Extension educators assist communities in building a 5.07 

sense of responsibility for wise use of natural resources 
in the context of global trends. 


.62 


R 


Faculty increase their expertise by interacting 5.07 

with faculty and scholars from other cultures. 


.47 


E 
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Human and physical resources are allocated to 
support the integration of international activities 
in the overall institution effort. 


5.07 


.47 


E 


Opportunities for international experiences are provided 
for administrators. 


5.00 


.55 


E 


The central mission of the Extension system includes a 
commitment to international education. 


5.00 


.55 


E 


Professional improvement activities increase 
activities increase knowledge of global issues. 


4.93 


.47 


E 


Extension is involved with international development 
activities. 


4.93 


.92 


E 


Local business persons are trained for participation 
in international markets. 


4.93 


.62 


R 


Specific groups (i.e. commodity groups) are 
targeted for public policy education on global 
decision-making. 


4.86 


.66 


R 


The organization's best junior faculty/agents 

are identified to participate in overseas assignments. 


4.86 


.36 


E 


Administrators engage in experience which will 
internationalize their own professional lives. 


4.86 


.53 


E 


Regular encouragement/accommodation of 
visitation by scholars from other countries occurs. 


4.86 


.66 


E 


Proposals for international work are developed 
and funded. 


4.77 


.44 


E 


The organization's best senior faculty/agents are 
identified to participate in overseas assignments. 


4.64 


.63 


E 


Exchange programs with extension organizations 
in other countries are institutionalized. 


4.64 


.74 


E 


Rural clientele are targeted for educational 
programming related to the current 
international marketplace. 


4.64 


.74 


R 


Educational programs planned by Extension help 
clientele secure a better understanding of complex 


4.57 


.76 


R 
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worldwide issues. 



Extension educational programs offered to 4-H 
members help develop international awareness. 


4.57 


.76 


R 


Educational programs increase participant's 
understanding of other cultures. 


4.57 


.76 


R 


A committee(s) is established to guide 
internationalization efforts. 


4.57 


.65 


E 


Exchange programs with extension organizations 
in other countries are planned and conducted on an 
on-going basis. 


4.50 


.65 


E 


Training programs are provided for foreign immigrants 
living in the United States. 


4.50 


.52 


“r 


Urban clientele are targeted for educational 
programming related to the current international 
marketplace. 


4.50 


.65 


R 


Extension clientele interact with visiting scholars 
and students to become more globally aware. 


4.31 


.75 


R 



Scale: 0 = No Importance; 1 - Slight Importance; 2 - Limited Importance; 3 - Moderate 
Importance; 4 = Moderately High Importance; 5 = High Importance; 6 = Critical Importance 
Categories: E = Organizational Effort; R = Organizational Result 
Note: Round 1: N=14; Round 2: N=13; Round 3: N=14 
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ABSTRACT 

School-to-Work (STW) activities have often been conducted with little knowledge about 
what to include in curricula and what the impact should be. The study addressed these concerns 
and issues, building from the Turning Points framework that documents a mismatch between 
middle school curricula and young adolescents ’ needs. Questions focused on curriculum 
implementation and focus, benefits to students, and implementation issues and concerns. 

The research method was qualitative with interview protocols used to gather information. 
A call for comprehensive, long-term STW middle school locations resulted in nominations of 36 
middle schools representing 16 different states. Contact persons at 28 of the 36 middle schools 
were interviewed. At six of these 28 middle schools, interviews were also conducted with 3 to 5 
others including at least one principal, one counselor, and one teacher. 

Enhancing curriculum relevancy, better serving needs of at-risk students, enhancing 
student development, developing career awareness, supporting systemic change and reform, 
building community linkages, and improving transitions were reasons for establishing STW 
curricula. Some interviewees offered conceptual reasons for implementing curricula. Several 
saw the curriculum as an excellent fit with middle school philosophy. A small number of 
organizational/operational reasons for implementing STW curricula were mentioned. Curricula 
focused on five areas: career exploration and awareness, self awareness, contextual learning, 
service learning, and integrated themes. Regarding curriculum determination, several educators 
indicated a broad net was cast to capture curriculum content. The range of benefits provided to 
students underscored contributions STW experiences made to young adolescent development. 
Educators noted the curriculum enhanced students’ personal development in individual growth, 
self understanding, confidence, self-esteem, and motivation and responsibility to learn. 
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OVERVIEW 



Even though middle schools have existed for over 40 years, they are considered by many 
to be a relatively new phenomenon. And what motivation existed for establishing middle schools 
in the first place? Kindred, et al. (1976, 3-9) described the growth of the middle school 
movement as a function of six interrelated areas: dissatisfaction with the junior high school, 
changes in young persons’ maturity patterns, new educational ideals, developments in learning 
theory, innovations in educational methods and materials, and changes in society. 

At that time, junior high schools were viewed as not providing broad exploratory and 
transitional experiences appropriate to adolescent students’ needs and interests. Over the years, 
middle schools replaced junior high schools to the point where a relatively small number of 
junior high schools still exist (Epstein, 1990). However, the other five growth areas noted by 
Kindred et al. in 1976 continue to evolve and in doing so present a contemporary frame of 
reference for discussions focusing on what middle schools should and should not be. Young 
people continue to mature at more rapid rates and thus provide middle school educators with 
continuing instructional challenges (National Middle School Association, 1995). Different 
philosophies of middle school education continue to be discussed and debated (Dougherty, 

1997). Learning research has provided middle school educators with much valuable information 
and, concurrently, raised a number of questions about how middle school students’ needs should 
be met (for example, see Anderman & Maehr, 1994). Innovations, particularly in the computer 
and electronic communication areas, have been touted as being important for students to learn so 
they will be prepared for life in our technological society. However, many middle school 
students are not afforded opportunities to learn about these innovations (Becker, 1990). And 
societal changes, such as evolution of the United States from a national economic powerhouse to 
being part of a global economy, have raised questions about what work-related education if any 
should be included in the middle school curriculum. 

SCOPE AND PURPOSE 

This paper focuses on a subset of middle school education that connects with each of the 
evolving areas introduced above; School-to-Work (STW) opportunities in the middle school. 
Over the past decade, increasing numbers of senior high school educators have provided 
comprehensive and meaningful STW opportunities for their students. Unfortunately, these STW 
opportunities may be offered too late in some high school students’ studies to have much impact 
on them. By the 9th or 1 0th grade, many students have already become turned off to education 
and have decided to quit school or just comply with minimum requirements for graduation. 

Other students may not have received much parental and peer encouragement to study and/or do 
not view schooling as an avenue to future occupational and career success (Kennedy, 1996; 
Lichtenstein & Blackorby, 1995). In response to these and other concerns, a number of school 
districts across the United States have created School-to-Work opportunities for middle school 
students. Examples range from including career exploration activities in individual middle school 
courses to school- and school district-wide incorporation of School-to-Work opportunities in the 
curriculum. In some school districts, educators are providing middle school students with 




49 



31 



meaningful experiential learning related to occupations and careers (Schmidt, Finch, & Moore, 
1997). 



Although educators are gaining experience at implementing school-to-work (STW) 
opportunities in the middle school, these activities have been conducted largely on an ad-hoc 
basis with little knowledge about how and why they should be included in the middle school 
curriculum as well as the impact they are designed to have on students. This study was 
conducted to address these concerns and issues, building from the Turning Points (Carnegie 
Council for Adolescent Development, 1989) framework that documents a mismatch between 
middle school curricula and young adolescents’ needs. Within the middle school context, 
answers were sought to a series of questions that were posed to middle school educators who had 
implemented STW curricula in their schools: 



1 . Why have STW curricula been implemented including conceptual, organizational, and 
operational reasons? 

2. What are the foci of the curricula and how were they determined? 

3. What benefits do the curricula provide to students? 

4. What issues and concerns are associated with implementing STW curricula at the middle 
school level? 



FRAMEWORK 

Since 1963, when the middle school concept was first introduced by William M. 
Alexander, educators have been searching for ways to embrace it. Middle school theory surmises 
that early adolescents’ academic, social, and emotional needs are better served by an educational 
experience not found in the elementary or high school enviromnents. Some middle-level schools 
have been restructured and transformed to provide the appropriate learning environment that 
assists young adolescents to meet their potential, and provide a meaningful place for young 
people in an adult culture. Steps for accomplishing this change have been outlined in Turning 
Points: Preparing American Youth for the 21st Century by the Carnegie Council on Adolescent 
Development (1989), and have been widely disseminated as a national “blueprint” in 
restructuring efforts. These steps include the following: 

• “create small communities for personalized learning (small schools or small programs 
within larger schools) 

• create successful experiences for all students by eliminating tracking and promoting 
cooperative learning 

• give teachers and administrators decision-making power concerning curriculum and 
instruction 

• employ teachers who like, respect, and appreciate adolescents 

• employ teachers who are experts in teaching young adolescents 
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• improve academic performance through fostering health and fitness of young adolescents 

• encourage family involvement in the education process 

• cormect middle schools with their communities (p. 9)” 

The authors of Turning Points noted that middle schoolers need to become “socially competent 
individuals” who are able to cope successfully with everyday life. They need to believe that they 
have promising futures and the competence to take advantage of societal opportunities when they 
arise. 



Unfortunately, many middle schoolers “lose ground” academically during this period. 
Theory suggests the declining academic achievement that commonly plagues adolescents may be 
directly related to “the mismatch between the developmental needs of the students and the 
educational environment” (Mac Iver, 1989). In turn, providing young adolescents with a 
combination of both challenging and nurturing experiences in appropriate settings can strengthen 
the possibility of them becoming more effective academic and social participants. Substantial 
evidence can be found of a relationship between young adolescents’ perceptions of the classroom 
environment and their achievement and attitudes (Dougherty, 1997). 

It was emphasized in Turning Points (Carnegie Council on Adolescent Development, 
1989), that a tremendous mismatch existed between middle grade curriculum and the needs of 
young adolescents. As Felner, et al. (1997a, p. 521-522) stated. Turning Points has provided 
educators with “comprehensive and researchable constructs and exemplars of those constructs to 
undergird their reform efforts.” In fact, the eight major recommendations for reform provided in 
Turning Points (cited earlier) served as a framework for the long-term evaluation of a model for 
high performing learning communities (Felner, et al., 1997b). The Turning Points framework, 
likewise, served as a framework to guide this study; particularly regarding ways middle schools 
utilize school-to-work/careers curricula to reduce the mismatch between the middle school 
curriculum and students’ needs. 



METHOD 

The research method was qualitative, with telephone interviews serving as the 
information gathering process. Interview protocols were developed to gather in-depth 
information from middle school educators. Concurrently, locations where comprehensive, long- 
term STW opportunities were being provided for middle school students were identified. State 
STW coordinators, selected association representatives, and VocNet listserve subscribers were 
asked to nominate middle schools where comprehensive, long-term STW curricula had been 
established. Nomination criteria defined middle schools that (a) had a fully operational, 
successful school-to-work/careers program, (b) were graduating students from the program, (c) 
had effective linkages with high school school-to-work/careers efforts as well as the workplace 
and the community, and (d) had a long-term commitment to STW transition at the school-district 
level. 36 middle schools representing 16 different states were nominated. Contact persons 
available at 28 of the 36 middle schools were interviewed to gather detailed information about 
their curricula. The remaining eight schools either did not meet selection criteria or their 
representatives were unavailable to complete the interview within the time constraints of the 
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study. At six of the 28 middle schools, interviews were also conducted with 3 to 5 additional 
persons including at least one principal, one counselor, and one teacher. Taped information 
gathered during the in-depth interviews was transcribed for comprehensive analysis using The 
Ethnograph (Qualis Research Associates, 1990). This software provided us with the capacity of 
coding, grouping, and regrouping information according to predominant and underlying themes. 

FINDINGS 

Middle school personnel that were interviewed responded to questions concerning four 
areas: (a) reasons for the implementation of a middle school STW curriculum; (b) curriculum 
focus; (c) ways the middle school STW curriculum contributed to student success; and (d) issues 
and concerns that have been voiced about implementing a STW curriculum at this level. 
Examination of interviewee responses resulted in the categorization of interviewee responses for 
each question. Descriptions of these categories are presented. Selected responses made by 
principals, academic and vocational teachers, STW coordinators, guidance counselors, school 
administrators, and others are included to assist in describing the nature of each category. 

Why Implement School-to-Work Curriculum? 

Reasons given for implementing a middle school STW curriculum were distributed into 
seven categories including Enhancing Curriculum Relevancy, Serving At-Risk Populations, 

More Effectively Enhancing Student Development, Developing Career Awareness and Career 
Exposure, Supporting Systemic Change and School Reform, Building Community Linkages, and 
Improving the Transition to High School and Beyond. 

Enhancing Curriculum Relevancy 

The most frequently cited reason for implementing a middle school STW curriculum was 
to help students make a connection between school learning, the workplace, and future living. 
Several interviewees indicated that middle school students generally do not connect the 
classroom and the “real world”, and that middle level educators needed to find more creative and 
innovative ways to answer the question, “Why do I need to learn this stuff?” Interviewees 
indicated that in order to bridge the gap between schooling and the workplace, instruction and 
classroom activities must link career experiences and academics in relevant ways. Curriculum 
content must be viewed by students as having a meaningful and lasting connection to their lives. 
Interviewee responses also revealed that for curriculum to be relevant, it must be realistic. The 
STW approach was viewed as a way to “give the content a context” that “gives validity to what 
we do as educators.” One technology director stated: 

We really felt like, even at this age, that some kids were having some difficulty 
seeing the correlation between what was going on in the classroom and what was 
going on in the world of work. And some of the kids do not see the relevance for 
getting an education and studying the things that we study in our curriculum. 

Giving kids the opportunity to get into the workforce and see how various 
language arts skills and mathematics skills and science skills are being applied, 
really seemed to make sense. We felt that it would really give them a good 
understanding as to what education was all about. 
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Since the STW curriculum was viewed as being relevant, realistic and meaningful, many 
interviewees concluded that this approach would be motivating to the students. Motivation, or 
perhaps, lack of motivation, was frequently mentioned as a reason for implementing a School-to- 
Work curriculum. This curriculum approach was considered motivating because it was “hands- 
on”, innovative, active, interactive, and allowed students to visualize the end results. 

Serving At-Risk Populations More Effectively 

Many interviewees stated that their STW middle school programs were implemented, at 
least in part, as an alternative education program or a drop-out prevention strategy for at-risk 
students. The STW program was also viewed as a way to individually address learning styles and 
interests, ability levels, self-esteem issues, motivation issues, and remediation needs. It should be 
noted that several sites considered special populations to include not only the traditionally 
defined “at-risk” students, but also gifted and talented and nontraditional students. STW 
curriculum was viewed at several sites as a way to address gender equity issues and eliminate 
tracking in certain career paths. According to one guidance counselor: ^ 

It kind of grew out of a need of a way to communicate with some of the special 
needs students that we had here. We started to find that working with at-risk 
students using “learn and earn” and School-to- Work job experiences, job 
shadowing and so forth, had a lot of meaning for these students. This was an area 
that they were probably deficient in because positive working relationships 
weren’t always modeled at home for some of these students. 

Enhancing Student Development 

Two concerns were identified within this category: developing personal awareness and 
developing work-related skills. Personal awareness development was commonly cited as a 
reason for implementing a middle school STW curriculum. Interviewees referred to planning a 
purposeful and developmental STW program that addressed the areas of personal responsibility 
and dignity, self expression, self-knowledge, and self-discipline. One junior high school 
principal felt STW education allowed students to determine “where they see themselves in 
relation to others and as an individual.” 

Others focused on using STW education to enhance work-related skills such as, future 
goal clarification, informed decision-making, team building and problem-solving, collaboration, 
critical thinking, and leadership. One junior / senior high school principal believed STW 
education “allows kids to explore as many options as possible and helps them in making well- 
based decisions about what they want to be when they grow up.” 

Developing Career A wareness/Exposure 

Both career awareness and career exposure were frequently cited as key reasons for 
implementing a STW curriculum. Interviewees felt it was important for middle school students 
to “see what’s out there beyond the school walls.” Several interviewees commented on the need 
to expose students to a variety of occupations, not just the more common ones. In addition, 
interviewees were concerned that schools were not able to keep up with changes in the business 



world and that work-based opportunities could help by exposing students to “real world 
applications and experiences.” 

Interviewees at several sites also addressed developing the teacher’s career awareness as 
part of their STW agenda. Teachers were exposed to a variety of occupations and career 
curriculum options through summer externships, worksite visits, on-the-job experiences, 
shadowing activities, conferences, resource library materials, weekly meetings, and specific 
training sessions. These experiences were expected to impact the teacher’s instruction, 
curriculum implementation, and assessment practices. Out-of-school opporhmities were intended 
to “train teachers how to weave School-to-Work ideas into their daily curriculum” and “change 
the teacher’s way of thinking about relating instruction to the workplace.” 

Supporting Systemic Change and School Reform 

In addition to enhancing career awareness, interviewees saw the STW program as a 
vehicle to long-lasting and systemic change, a way to build a foundation for establishing a 
seamless, streamlined, comprehensive system. Some sites described their schools as “in crisis” 
and were using the STW approach as one of many strategies for initiating and sustaining school 
reform. Several respondents also discussed the need for a comprehensive, developmental 
guidance delivery system that included both career guidance and work-based learning activities 
for all students. Interviewees at several sites commented on attempts to revise, renovate, and 
refocus their career exploration programs to emphasize School-to-Work issues. The intention of 
this emphasis was on linking career experiences with both guidance and academics. 

Building Community Linkages 

The concept of community emerged as a dominant reason for implementing a middle 
school STW curriculum. In fact, several sub-categories noted in this category included (a) 
building community /business linkages, (b) addressing business/ industry/community concerns, 
(c) supporting local economic development, and (d) encouraging student participation in the 
community. 

Strong business and industry linkages were viewed as a vital part of the STW 
curriculum since worksite experiences and workplace understanding are essential 
ingredients for future success. Interviewee responses suggested that linking with the 
community was a two-way process. Schools attempted to address community, business, 
and parent concerns by surveying and assessing their needs and forming task force 
committees and partnerships to help determine avenues to successful improvement. Some 
sites tried to “align curriculum based on research from employers.” However, in return, 
schools expected the community and business members to become actively involved in 
the education of their young people. The business community needed to “help prepare 
students for the workplace, address the skills not being addressed in school, and make 
educators aware of workplace needs.” Many interviewees viewed the business 
community as willing but apprehensive participants in this process. 

Linkages were developed primarily in response to concerns voiced by employers 
and community members in neighboring areas. A STW curriculum was seen as 
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addressing the needs of very economically depressed areas, often preparing students for 
more skilled jobs since the local area had lost its unskilled employment base. 

In other instances, entrepreneurship was encouraged as one way to provide local work 
experiences for the students, bring new businesses into the community, and encourage graduates 
to stay in the area or return to the area to live. Entrepreneurship was also seen as a viable way to 
encourage students to expand their career options and aspirations, or as one STW program 
coordinator stated, “to think beyond I want to be a farmer.” 

Community involvement and service were frequently cited as reasons for implementing a 
STW curriculum at the middle school level. School personnel at several sites felt it was 
important to “help kids understand who they are in a community and to develop a sense of place 
in that environment.” Interviewees commented that middle school students often feel people 
don’t care about them and that they needed to consistently see that those in the community cared 
about their futures. The STW curriculum was viewed by one rural site community education 
director as “an opportunity to learn IN the community and contribute TO the community through 
community development projects.” In addition, STW can “give the community a more real 
perception of what goes on in schools.” 

Improving the Transition to High School and Beyond 

Several interviewees indicated that middle level educators must help to ease the transition 
into high school by providing a “continuum of services that link and build on previous 
experiences” and assisting students in “speaking the language of the high- school.” Middle school 
personnel discussed becoming partners with the high schools in a long-term process, indicating a 
need to examine how middle school activities and learning can influence high school decisions. 

A guidance counselor commented; 

I think the middle school [STW] curriculum really allows students, while they’re 

in middle school, to get a good sampling of everything that’s offered in high 

school so they can base their decisions on first-hand knowledge and experiences 

rather than hearsay or reading it in a booklet. 

Middle school was seen by many interviewees as a time when serious decisions about the 
immediate future needed to be made. In some cases, eighth graders were required to choose a 
high school based on its occupational focus or college preparation program and apply for 
admittance. Others needed to enter high school having identified a career cluster of interest and a 
four year plan of study. 

A majority of respondents indicated their STW programming was an effective alternative 
to traditional tracking because it increased the students’ achievement and exerted a more positive 
impact on the personal development of students with learning difficulties or delays. STW 
curriculum was seen as more readily adaptable to the pace and level of instruction needed for 
certain individuals. A STW coordinator commented on the ability of this curriculum to “target 
the actual needs of students and therefore, motivate them more effectively.” 

Although middle leVel educators interviewed saw the importance of preparing students 
for high school, most indicated a wider reaching influence. Interviewees indicated that STW 
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programs at the middle school level would develop students who were better prepared for post- 
high school education and careers in the future. An executive director of professional technical 
education stated that his district focused on providing a “smooth transition for students from 
school to school, school to post secondary opportunities, and school to work”, and that they 
intended to assure their students “learn more about the requirements, skills and benefits of a full 
range of career options and opportunities to help prepare them for the future.” 

Preparation for future work was frequently cited as a reason for implementing the STW 
curriculum. For example, one rural school principal insisted, “We have been too academically 
inclined. We must look at each student as a future worker.” Some respondents specifically 
addressed the need to prepare students to live in a technical world and to “provide training for 
the high tech workplace.” STW education was seen as an avenue to expose students to a wide 
range of technological careers and encourage interest in technology opportunities. 

Conceptual Reasons for Implementing the Curriculum 

Over half of the interviewees had no known conceptual reasons for implementing the 
STW curriculum. However, several respondents referred to the middle school philosophy and the 
Carnegie Turning Points as their conceptual basis for adopting a middle school STW curriculum. 
Some interviewees indicated that STW educational reform would advance the middle school 
agenda by “embedding basic skills into a thought-provoking curriculum” and “providing a 
holistic approach to the child.” At some sites, interviewees saw the STW initiative and the 
middle school philosophy as “a total fit” for this age group since their real task was to help the 
student transition from a child to a young adult. A science and technology magnet school 
principal explained the alignment between the two. 

It [STW curriculum] is very interactive, synergistic, and relevant to “real life”. It 
opens doors to new ideas, builds a full range of communication, uses multiple 
intelligences, develops thoughtful citizens, provides opportunities for service, 
accommodates special needs, assesses strengths and weaknesses, produces life 
long learners, and focuses on a moral and democratic society. Isn’t this what the 
middle school philosophy states? 

Some questioned the compatibility of the two concepts! One rural site principal 
commented, “The liability of the middle school philosophy is you become more elementary in 
nature. The needs of adolescents are just human needs. It’s never too early to address future 
needs.” Another rural school principal stated, “School-to-Work and Turning Points are not 
congruent. Turning Points philosophy de-emphasizes academics and works on affective 
behaviors. We don’t buy into this.” 

In addition to Carnegie’s Turning Points, several interviewees cited specific sources as 
conceptual reasons for implementing the School-to-Work curriculum. These included the 
SCANS competencies. National Career Development Guidelines, local and statewide learning 
standards for excellence. Equity Outcome Guidelines, Microsociety concepts, and state 
educational reform acts. 
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Organizational/Operational Reasons for Implementation 

One third of the interviewees noted organizational or operational reasons for 
implementing a STW curriculum at their site. Interdisciplinary teaming was most often cited as 
the organizational reason for implementing the STW curriculum at the middle school level. The 
teaming concept allowed for greater infusion of career materials and information across the 
different teaching areas. A suburban school principal saw teaming as a way to “encourage career 
infusion and integration by teachers.” Some sites referred to the career cluster organizations and 
block scheduling at area high schools as reasons for organizing their curriculum in a similar or 
complementary fashion. Others felt the STW curriculum supported the nature of a small school 
or school-within-a-school (academy) structure. 

The Curriculum Focus 

A review of interviewee responses concerning the curriculum focus revealed five 
categories of responses. They included Career Exploration and Awareness, Self Awareness, 
Contextual Learning, Community-based and Service Learning, and Integrated Themes. 

Career Exploration and Awareness 

One of the main focuses of the STW curriculum at the middle school level was career 
exploration and awareness. Curriculum activities and experiences were designed to “catch the 
interest of the students” and “expand their horizons.” Students were encouraged to evaluate their 
individual interests and abilities and build goal setting and decision-making skills through career 
exploration participation. Interviewees indicated that students were carefully guided through 
these developmentally appropriate experiences that focused on “exploring the world of work.” 

Self Awareness 

Student self awareness was also frequently mentioned as a curriculum focus. According 
to one academy principal, participation in the STW exercises would “create awareness of 
individual talents and build generic skills such as, problem-solving, decision-making, 
negotiating, finding resources, and working with others.” 

Contextual Learning 

Some interviewees felt strongly that the focus of the STW should be on real life 
community and workplace experiences rather than simply exploration. They indicated that the 
information and workplace skills learned in the classroom needed to be applied in actual 
workplace settings, in other words, to “connect basic learning with practical application.” A 
district administrator stated, “the School-to-Work implementation focus is the actual fusion of 
related skills into the curriculum and the actual practicing of those skills in career experience 
opportunities.” 

Community-based and Service Learning 

Several sites indicated community-based learning and/or “service learning” as an integral 
part of their STW curriculum focus. Most interviewees viewed community-based learning as a 
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way for students to build respect for themselves and acquire a stake in their community. 
Community-based activities such as clean-up projects and community safety campaigns. 
Interviewees indicated service learning activities helped to “develop a sense of civic 
responsibility” and “channeled energy into helping solve local problems”. Service learning 
activities were integrated into the academic curriculum and enhanced what was taught in school 
by extending student learning beyond the classroom. A district executive indicated that her 
school focused on service learning because it “encouraged volunteerism as a way to explore the 
reality of jobs.” 

Integrated Themes 

Many of the respondents indicated that STW education itself provided the focus for their 
academic curriculum. STW was not viewed as a separate curriculum, but carefully embedded 
within their daily routines. STW education was seen as a way to “reinforce the classroom 
curriculum” and “focus on continuous progress.” One principal, referred to his school’s 
curriculum as having a “school-to-life focus with a very integrated scope and sequence.” Some 
sites developed specific curriculum themes that heavily focused on STW content. For example, 
technology, medical careers, legal careers, economics, manufacturing, and entrepreneurship. 

Determining the Curriculum Focus 

Interviewees discussed several ways in which the STW curriculum focus was initially 
determined in their locality. These processes included faculty brainstorming, student input and 
selection, area needs assessments, community “conversations”, trial and error, individual teacher 
initiatives, and soliciting advice from those localities with past experience. These selection 
processes ranged from simple, unstructured in-house discussions to highly structured 
community-wide efforts. A rural site administrator discussed how themes emerged from a 
number of activities designed to discover a community’s concerns and priorities. 

... People were just talking a lot about what it is we need to do to create good 
teaching and learning environments for students. A lot of our priorities were 
changed within our district through a variety of these conversations and themes 
really have emerged in part from that, and part from individual teacher initiative 
and various expertise. We didn’t adopt it from anywhere else. It was an internal 
organic process. 

Many of the site respondents indicated that the curriculum focus or themes were 
constantly “under construction” and “always evolving.” Efforts were being made to keep 
up-to-date and anticipate possible roadblocks and new avenues of interest. 

BeneHts the Curriculum has Provided to Students 

An examination of interviewee responses concerning the benefits their middle school 
STW cumculum had provided to their students revealed seven categories of responses. These 
categories included Connected Classwork and the “Real World,” Enhanced Student Personal 
Development, Improved Student Behaviors and Attitudes, Strengthened Business and 
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CommunityLinkages, Strengthened Existing Academic Curriculum, Served Special Populations 
Needs, Prepared Students for the Future. 

Many of the benefits related directly to reasons previously mentioned for having 
implemented the STW curriculum. In other words, students appeared to have benefited in ways 
that educators had hoped they would. However, some of the other benefits mentioned such as, 
“improved student behaviors and attitudes” and “strengthened academic curriculum”, were not 
specifically mentioned as reasons for having implemented the curriculum. These seven 
categories are not mutually exclusive and thus may have some overlapping characteristics. 

Connected Classwork and the “Real World” 

A majority of respondents indicated that the number one benefit derived from the STW 
curriculum consisted of opportunities for students to apply what was learned in the classroom to 
real life situations, settings, and problems. STW activities and experiences were viewed as 
beneficial because they were “realistic”, “relevant”, and “meaningful” to the students. This 
“connection” with reality was seen as not only between the classroom and the workplace, but 
between the classroom, the community, and students’ future goals. One inner city school 
principal shared her overall view. 

They [students] begin to see the connections between what they do, what they 
learn, what they produce, and what opportunities are available to them. The major 
issue would be that there’s real life application; real-life application of values, 
real-life application of what do we think a good society ought to have, and how do 
we support that, and a real-life application for the various academic areas that one 
is studying. 

Enhanced Student Personal Development 

Interviewees revealed that one of the major benefits of implementing a STW curriculum 
for middle schoolers is that it appealed to the students at their developmental level. This 
curriculum had more “personal meaning” for the students. One inner city school principal 
elaborated on this point, “It empowers them, and what we know is that children at this age need 
to feel that their voices are heard. It also addresses the issues that are germane to a young 
adolescent.” This opinion was echoed by a rural site administrator. He said: 

I think junior high students are particularly . ...looking to try and make sense of 
what it is they’re studying and of their world and their place in the world. They 
are really eager to connect with the real world and some of the issues that are 
going on in the world and the stories that are happening in the world. So 
curriculum projects, project-based learning, hands-on learning, service learning, 
learning connected to the community in some way are really important to them. 

Several respondents indicated that the STW curriculum helped the students to: (a) build 
meaningful relationships with adults and peers; (b) improve in their ability to work as members 
of teams; (c) grow in self understanding, sophistication, patience, self-esteem and confidence. 
Students were able to “reflect on and think critically about their experiences” and “recognize the 
intrinsic value of education and of success and the work ethic.” 
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Improved Student Behaviors and Attitudes 



Frequent references were made to the STW curriculum’s direct and positive effect on 
student behavior. Interviewees had witnessed an increase in active learning behaviors such as, 
asking questions, participating in discussions, task engagement, and attendance. A few schools 
specifically stated that they had seen a “steady rise” in standardized test scores for those students 
involved in the STW curriculum. Improved behavior and attitude adjustment were also evident to 
educators because of the decrease in truancy, delinquency and discipline referrals, social service 
interventions, arrests, insolence toward adults, and general “acting out.” Positive behavior and 
attitude changes were also witnessed by school personnel outside the classroom. 

Strengthened Business and Community Linkages 

Interviewees indicated businesses and industries that had invested time, personnel, and 
funds in the career component of the STW curriculum were beginning to re-evaluate their initial 
apprehensions about the program and middle school students. Interviewees also noted that over 
time businesses, industries, and community agencies offered more extensive and varied first- 
hand worksite experiences for both the teachers and the students. In addition, businesses 
provided more reliable and more involved adult worker role models and mentors for the students. 
Participating businesses \vere also eager to recruit new businesses into the STW program. 

The STW curriculum was viewed by many respondents as beneficial because it provided 
a venue for parents, teachers, and community members to “talk the same vision and goals.” The 
curriculum was seen as a way to bring the community into the schools and the schools into the 
community. One rural site administrator extended this idea to include the building of a 
participatory culture. He said: 

I think there’s a cultural change here, the way we think about our community. I 
think there’s this idea of community building within the school and being active 
players in a community building process. I think that’s really the heart of how our 
teachers and a growing number of parents and students are starting to think about 
school and our community. People are telling us there’s something that’s 
working there. I don’t think it’s just the curriculum, it’s a culture that’s emerging. 

Strengthened Existing Academic Curriculum 

One of the benefits commonly expressed by the interviewees was how the STW 
curriculum focus had strengthened the academic curriculum currently in place in the schools. The 
STW focus was said to incorporate traditional educational experiences and values with 
innovative techniques and methods. One academy principal referred to the curriculum as a 
superior way of “integrating theory, practice, and hands-on activities.” Interviewees indicated 
that the STW focus benefited students by “tying our current curriculum to something that’s . 
important to students” and because “it’s a curriculum that makes sense to the kids.” Teachers 
from several schools commented that students were drawn to this STW curriculum focus because 
of its “realistic”, “different”, “more involved”, and “experiential” approach to learning'. Other 
interviewees recognized the STW curriculum as having benefited students by providing an 
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increased depth and breadth of information and experience, when compared to the traditional 
curriculum. 

Served Special Populations Needs 

Another benefit commonly cited by interviewees was that the STW curriculum assisted 
educators in providing a more appropriate and comprehensive curriculum for certain special 
populations, specifically educationally or economically “at-risk” students and minority students. 
A career counselor discussed several of the benefits “at-risk” students receive through 
participation in a STW middle school curriculum. 

I think there are some benefits for ....at-risk kids. A lot of at-risk kids have 
relatively small spheres of experience and [the STW activity] gives them an 
opportunity to get out into places that they’ve never been before, to see what other 
people are doing. I think it gives them an opportunity to see models in the 
community that they don’t typically see. 

Other interviewees commented on the curriculum’s positive effect on minority students in their 
schools. The STW curriculum was viewed as a way to “pull minority populations together” and 
to provide minority students with “new ways to view their futures.” One administrator of a 
predominantly minority school stated, “...the kids learn about each other’s background and how 
to be sensitive.” 

Prepared Students for the Future 

Interviewees indicated in their responses that middle level students benefited from a STW 
curriculum because it better prepared them for high school, post-secondary education, the 
working world, and adult living. Responses indicated that the STW curriculum prepared students 
for these future settings and situations by “opening their eyes” and “opening their minds” to all 
the options and occupations available and the “path” commonly taken to achieve certain desired 
outcomes. The STW curriculum was seen by some interviewees as a workforce preparation 
program but not in the sense of having middle level students decide on a future career at this 
young age. Instead, the curriculum was meant to expose students to many different careers and 
“help and guide them in eventually choosing a career based on their values, personalities, and 
abilities.” 

Several interviewees from larger cities emphasized the need to prepare students at a 
younger age (middle school level) because eighth graders in their districts were required to select 
and apply to individual high schools based on their personal career interests and the schools’ 
career themes. A city district administrator explained how middle school students were 
benefiting from their career pathway/STW model. He said: 

...they [middle school students] enter high school with a better focus of what they 
are interested in studying. They are seeing that accountability and teamwork are 
critical to work in the high school instructional environment. Regardless of 
whether these students go on to college, to further training, or enter the workforce 
directly, middle school students enter the high school being prepared to succeed 
by working with “real world” application in their education. 



Implementation Issues and Concerns 



Interviewee responses on issues and concerns that have been voiced about implementing 
a STW curriculum at the middle school level could be grouped into four categories of responses. 
They included Participant “Buy-In,” Program Logistics, Resources, and Program Quality and 
Outcomes. 

Participant ‘‘Buy-In ” 

Interview responses indicated that adults who would potentially be participants in the 
STW curriculum effort (teachers, parents, guidance counselors, community members, and 
business representatives) needed to be “sold” on the idea of STW for middle level students. 
Students appeared to be the only stakeholder group that did not exhibit initial resistance to and 
apprehension about the curriculum. Classroom teachers were overwhelmingly the most reluctant 
curriculum participants. Many classroom teachers originally viewed the STW curriculum as a 
“fad” or “just one more thing” but in due time came to recognize its value for their students. 
Those educators “selling” the curriculum needed to show reluctant teachers how it was an on- 
going, integrated process that would change the way they taught. Teacher concerns lessened as 
they became more familiar with the integration aspect of the curriculum. 

Another issue raised concerning teacher “buy-in” was the concept that the STW 
curriculum was intended to be integrated into all subject areas and therefore was meant to 
be implemented by all teachers, academic and vocational. Traditionally, this career focus 
was strictly a vocational teacher’s or counselor’s domain and responsibility. Interviewees 
indicated that academic teachers needed additional convincing that career concepts were 
applicable and valuable to their class content and goals. 

Some sites had STW programs that encompassed only a portion of the students in a 
school. This arrangement led to another teacher “buy-in” issue. Teachers working within the 
STW programs were often specially chosen for their positions and were enthusiastic about the 
career integration notion. However, teachers working outside the program were often unfamiliar 
with the innovative methods used to stimulate learning and resented the commotion it caused in 
the school environment. 

While the majority of interviewees indicated that parents of students in the STW 
curriculum were pleased with its conception and implementation, respondents did acknowledge 
some minor concerns on the part of individual parents. Some parents were concerned with the 
number of field trips and time spent outside the school building. Others were concerned about 
the possibility of the STW / careers programs “tracking” their children into a particular career 
path at too early an age. This concern appeared to be more prevalent in schools that did not 
provide STW opportunities for all students. 

Community members voiced some opposition to the teaching and learning methods used 
in the STW curriculum. Interview responses revealed that a majority of the community members 
were very supportive of the out-of-school student experiences and ingression into the community 
and business world. However, other community members were adamant about maintaining the 
“status quo” and retaining “the way it was.” Interviewees revealed that there had been some 
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“detractors” who felt the STW curriculum was strictly a high school and beyond initiative. These 
community members believed the time and resources used to put students in the workplace 
would be better spent when the students were older. Several interviewees commented that they 
had anticipated a greater outcry against the STW initiative from certain groups of individuals in 
their community. 

Interviewee responses indicated that members of the business community were very 
supportive of the concept behind STW for the middle level student but very apprehensive about 
the student’s educational and emotional maturity levels. Most of the business participants had 
never worked with middle school students and did not know what to reasonably expect. Business 
members were also very concerned about the worksite safety and liability issues surrounding a 
program for underage “workers.” 

Program Logistics 

Many of the concerns voiced by curriculum participants centered around the practical 
day-to-day implementation of the curriculum. Interviewees’ concerns focused on four sub- 
categories: time, scheduling, legalities, and transportation. 

The majority of interviewees commented on the “time-consuming” aspect of 
implementing a comprehensive STW curriculum. It was seen to be time consuming from both 
the administrative and teacher viewpoint. An inner-city site principal highlighted this concern. 

It’s very labor intensive on the part of the adults because you are taking on so 
many other roles and functions. Teachers who are preparing for their academic 
arena also have to be preparing for the School-to-Work arena. That’s very 
intensive. 

Activity and work-experience scheduling was a concern that appeared frequently in the 
interviews. As one academy principal suggested, “The whole scheduling arm of this, it’s not 
impossible but it’s certainly an issue that needs one’s attention because if it’s going to be real, 
then it can’t be Mickey Mouse.” Some interviewees indicated that they were “hamstrung” by 
their schedules, particularly those working in schools functioning within the traditional eight 
period school day. Some people at these sites indicated that the schedule limited their options 
and fostered departmentalism rather than interdisciplinary activities. Although block scheduling 
was not seen as a reason for establishing a STW program, it was frequently cited as being 
“valuable”, “beneficial”, and “helpful”. Interviewees claimed block scheduling supported more 
integrative curricula, flexible programming, in-depth exploration, extensive community and 
workplace visits, and in-school speakers, films, and workshops. Block scheduling also gave 
schools more time for teacher development activities such as inservice education and new 
materials review. 

The legalities of implementing a STW curriculum that included worksite visits or on-the- 
job training (internships, apprenticeships) were commonly cited as concerns. The legal concerns 
most frequently mentioned were business and school liability, student safety, and child labor law 
issues. In addition, transporting students to jobsites, on field trips, and to area high schools in a 
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safe and timely fashion was viewed as a significant curriculum implementation challenge by 
many of the respondents. 

Resources 

Resources, both financial and human, were frequently cited by interviewees as the 
number one issue or concern in implementing a middle school STW curriculum. Interviewees 
were particularly concerned with two financial situations: maintaining the curriculum “as is” 
after grant funds dried up, and locating funding for the expansion of a component of the STW 
curriculum (e.g. internships). Several interviewees commented on trying to maintain and build 
on a non-mandated program during a time of “reduced school spending” and “district budget 
cuts.” A city school principal explained how finding additional funding sources was essential to 
running a successful STW program. She said: 

Resources are the key. What we’re trying to do could not be done by us alone on 
the traditional school budget. I think more people are cognizant that we have to 
have additional resources, and that it’s going to take more than one or two people 
writing grants if we are to be fully successful in what we’re trying to do. 

Resources are constantly an obstacle.. ..in order to do project-based learning, 
which is critical to School-to-Work, you can’t simulate a career using textbooks in 
the classroom. 

Interviewees noted that personnel needs were an important area of concern. Finding 
qualified and willing business people to oversee and supervise students on the worksites and act 
as mentors and role models was reported to be “a challenging aspect of the STW program.” In 
addition, convincing business partners and community members to continue supplying the local 
STW program with speakers, mentors, job coaches, etc. was viewed by some interviewees as a 
possible future concern. One city school principal described the need for business participation in 
the STW effort at her school as “a constant challenge” and “an on-going focus.” Interviewees at 
several of the sites were concerned that certain essential in-school personnel positions, such as a 
STW coordinator position, would be eliminated when grant money disappeared, and that the 
classroom teachers would be burdened with the responsibilities of that position. 

Program Quality and Outcomes 

Many of the interviewees mentioned concerns about improving the STW classroom 
instruction and expanding the work-experience opportunities each year. Other interviewees were 
concerned about the consistency of quality in student work produced in the STW program. 

The measurement of growth and progress by analyzing changes in individual student test 
scores is a common practice in many school systems. A few of the interviewees were concerned 
because the progress students had made in the STW program was not necessarily evident to 
teachers outside of the STW program or in standardized test scores. 

Resolution of the Issues and Concerns 

Overall, interviewees indicated that their issues and concerns about implementing and 
maintaining a quality STW curriculum for middle school students were “challenging” but 



“workable over time.” According to one principal, “There’s always going to be a challenge. 
There’s always going to be a personality. It’s just like the real world, there’s always going to be 
something to overcome, but that’s just part of it.” Interviewees also suggested that the “trials and 
tribulations” they had to work through were typical for any new or innovative program 
introduced to a school or school system. Some concerns, such as transportation availability, were 
viewed as “on-going” and “expected”. One community education director explained the process 
followed to resolve issues and concerns in his school. 

My sense is that it’s just going to take some time. I think there’s just a lot more 
conversation and understanding and group plaiming that we have to do to really 
take it to the next level. That will come. I don’t think it’s insurmountable, but we 
are going to have to rethink the basic ways we think about school and see how 
good we are at doing that. 

DISCUSSION AND IMPLICATIONS 

It was anticipated that educators would include enhancing curriculum relevancy, better 
serving needs of at-risk students, and enhancing student development among their reasons for 
curriculum implementation. The remaining themes (developing career awareness and exposure, 
supporting systemic change and school reform, building community linkages, and improving the 
transition to high school and beyond) were less obvious in the literature but appear to be of no 
less importance. All reasons were to some extent a function of school context. 

About half of the middle school educators interviewed offered conceptual reasons for 
implementing their STW curricula. Some persons referenced Turning Points (Carnegie Council 
on Adolescent Development, 1989) and/or general middle school concepts as a foundation for 
curriculum development efforts. Several educators saw the STW curriculum as an excellent fit 
with the middle school philosophy of assisting students to transition from child to young adult. 

Interviewees mentioned a small number of organizational and operational reasons for 
implementing STW curricula in the middle school. Interdisciplinary teaming, which was 
discussed most frequently by educators as an organizational reason for implementing STW 
curricula, is quite visible in the literature. However, it is not clear who the members of these 
teams should be. 

Collectively, middle school educators indicated their curricula focused on five different but 
interrelated areas (career exploration and awareness, self awareness, contextual learning, service 
learning, and integrated themes). It was in this area where STW curricula appeared to differ most 
from curricula advocated in the general middle school literature. However, the actual difference 
is quite subtle. Regarding curriculum determination, implicit in the literature is a view that 
educators are the source of content knowledge and organization for middle school curriculum 
development. In contrast, several educators indicated that at their schools a broad net was cast to 
capture content for inclusion in the curriculum. Through a range of approaches, educators 
brought a real world focus and view into the curriculum. 

Interviewees described a broad range of benefits STW curricula provided to their students. 
Comments underscored contributions of STW experiences to middle school student 
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development. Middle school educators noted the middle school STW curriculum enhanced their 
students’ personal development in the areas of individual growth, self understanding, 
confidence, self-esteem, and motivation and responsibility to learn. 

Several suggestions are offered for consideration by those interested in more fruitful 
collaboration between School-to-Work curricula and the middle school agenda. As a starting 
point, consider the direction STW opportunities in some middle schools appear to be taking. As 
described by educators in exemplary middle schools where STW curricula are being provided to 
students, 

• these students can prepare for their futures in addition to satisfying their current needs. 

• teaching and learning focus on both the educational process and its outcomes. 

• every educator in the school can team with each other as well as with community and 
workplace representatives to provide students with authentic learning experiences. 

• the context for teaching is proactive and dynamic rather than reactive and static. 

• the curriculum can be developmentally responsive to students and concurrently provide them 
with a wide range of opportunities such as career exploration and awareness, contextual 
learning, service learning, and integrated learning themes. 

Thus, there appears to be a clear connection between what the middle school literature says 
middle schools should do and what a number of STW-oriented middle schools are doing. Even 
though STW opportunities in the middle school may not be a mainstream focus for middle 
school professionals, these opportunities have the potential to meet students’ developmental 
needs in new and exciting ways. It is therefore important to better understand and document 
exemplary STW opportunities that are occurring in many middle schools across the United 
States so their successes can be shared with other middle school educators. 

It likewise appears that middle schools where STW opportunities are being provided to 
students may indeed be exemplars of best practice as envisioned in the middle school literature. 
Broadly-based teacher teaming, extensive linking with the community, providing students with 
opportunities for contextual learning, enabling students to explore the real world, and providing 
students with meaningful development experiences are all suggested or implicit in the middle 
school literature and can all be accomplished within a STW opportunities framework. 
Descriptions about STW opportunities that can be provided to middle school students and their 
potential value must be communicated to the middle school educator community. Middle school 
educators should have access to this information before they begin to implement major 
curriculum changes. In fact, the STW curriculum may be exactly what a number of middle 
schools really need. 
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COMPARING THREE AUTOMOTIVE PARTNERSHIP PROGRAMS AT THE 
COMMUNITY COLLEGE LEVEL WITH THE SCHOOL TO WORK MODEL 



Michael L. Klyde 



Leonard Albright 



Introduction to the Study 

The automotive service industry faced increasing consumer demands for quality repair 
work and a shortage of qualified new technicians. Many long time technicians have also been 
retiring from the profession, creating additional need. These factors are contributing to the 
National Automotive Technicians Education Foundation (NATEF) prediction for hiring an 
additional 60,000 technicians by the year 2000 (NATEF, 1998, p. 1). The complex electronic 
and computer technology found on today’s vehicles have placed greater demands on educational 
institutions to insure automotive technology students acquire skills directly relevant to becoming 
competent auto technicians. 

Partnerships between automotive businesses and community college systems represented 
one method to educate and prepare students for work as automotive technicians. Since the mid- 
1980’s, General Motors (GM), Ford, Chrysler, Toyota, and Nissan have developed partnerships 
with community colleges throughout the U.S. These partnerships provided school and work- 
based student education. The automotive instructors established relationships with local auto 
dealers for student placement in work-based learning activities, participated in corporate training 
to keep pace with the latest technology, and the colleges received vehicles, tools, and equipment. 

Since passage of the School-to-Work Opportunities Act (STWOA) in 1994, research on 
various types of partnership programs have grown in number, including research identifying 
indicators of successful school-to-work partnerships. Although many automotive partnerships 
are more than 10 years old, research specific to these partnerships has been scarce. 

Statement of the Problem 



Community college automotive partnership programs have faced many challenges. 
Critical elements on which the success of a program hinges include: maintaining enrollment; a 
strong dedicated faculty; relevant, practical experience for the students; and cooperative business 
partners. Successful programs provide students with the cognitive, behavioral, and affective 
elements necessary to transition into a full time career in the automotive profession. The 
business partner provides the student with a real world setting to practice these skills. The 
student benefits in knowing from the start that their course work has real world relevance. The 
business partner benefits by having a steady supply of new talent to preview while the student is 
in school. The college benefits from donations made by the business partner, training provided 
to college faculty by the partner, and having a program that attracts local students. 

Research work in the early 1990s has identified indicators present in a broad spectrum of 
successful school-to-work partnership programs. An example of this research was A 
Benchmarking System for Tech Prep, from the Center on Education and Work at the University 
of Wisconsin, Madison (Connell, Dungan, & Mason, 1995). This research developed a self- 
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assessment instrument for tech prep and partnership programs. Yet, very little research 
specifically focused on automotive partnership programs has been done since passage of the 
School-to-Work Act in 1994. 

An examination of successful school-to-work automotive partnership programs at the 
community college needed to be conducted. The intent of this research was to compare 
indicators of success present in automotive partnership programs with those identified in 
previous research of other school-to work (STW) programs. 

Statement of the Purpose 

The purpose of this study was to identify indicators contributing to the success of three 
school-to-work automotive partnership programs at community colleges in the Southern 
California area. Characteristics of a successful program include: Formal instructional plans that 
link the work place with college course work; an operationalized commitment that was carried 
out by the college and business partner(s); completers of the program who reached their initial 
goals for school and work; and innovative approaches to deal with curriculum, administration, 
and partnership issues. The research involved three Southern California community college 
automotive partnership programs. The partnerships are comprised of a business partner (Ford, 
General Motors and Toyota) and one of their community college sites. For this study, the 
partnership program managers at Ford Motor Comply, General Motors Corporation, and 
Toyota Motor Sales of America, Incorporated were contacted and asked to identify their 
strongest partnership college site in the Southern California area. The School-to-Wprk Self- 
Assessment checklist (instrument) used, with permission of the University of Wisconsin, was 
provided to community college faculty involved in the partnership. Minor word changes to the 
instrument were made to improve clarity for the subjects in the community college partnership 
program. The subjects included the program coordinator, instructors, department dean, and 
counseling staff. A group interview was conducted with partnership program students at each 
site to obtain their input on indicators of program success that were directly relevant to them. 

Site visits were made to view facilities and observe each program in action. Follow-up 
interviews with the partnership program coordinator(s) were made for clarification of data. The 
research proposes to answer the following questions: 

1 . Which indicators of successful partnerships identified from previous research are 
common to these programs? 

2. Which indicators of successful partnerships identified from previous research are not 
present in these programs? 

3. Are any unique indicators identified in these programs that have not been cited in 
previous research? 

4. Are there significant differences in each program and if so, how do they contribute or 
detract from program success? 

After data collection, answers were synthesized from participant responses, comparison of the 
responses among the research participants, and previous research in this area. 
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Design of the Study 



This research was a study of three automotive partnership programs located in Southern 
California. Ford, General Motors (GM), and Toyota were selected as the business partners 
because of the longevity and commitment of their programs. The college sites were selected 
based on input solicited from the Ford, GM, and Toyota corporate managers responsible for the 
Southern California partnership programs. Each nominated commimity college was then 
contacted for willingness to participate. All verbally agreed, leading to formal paperwork and 
Institutional Review Board approval of California State University, Long Beach. 

The research methods included adaptation of the self-assessment instrument developed 
by the University of Wisconsin, Madison. College faculty and staff who work with the 
partnership program were asked to complete the survey instrument. 

Site visits to observe program elements and conduct group interviews with partnership 
program students provided additional insight. The use of three sites allowed the research to gain 
insight into the elements that are similar and those that are unique between them. The research 
data were reviewed individually and collectively. Although it was not a primary purpose, it was 
useful to discuss the similarities and differences between the three programs as they related to 
the presence or absence of indicators. Using three sites helped eliminate problems that may have 
occurred using only one site. Examples of problems were: imique characteristics present at one 
site but not common in these programs, including a site that was not as successful as the business 
partner believed, and making recommendations and generalizations based solely on one site. 

The sample included community college staff involved with the partnership programs of 
Ford, GM, and Toyota, and currently enrolled (students) in these programs. College personnel 
included the dean of technology, the head instructor or program coordinator of each program, 
other instructors who teach automotive classes in the program, and coimseling staff that support 
the automotive department. The desired students for the research were those who were within 
two semesters of graduating. 

The survey instrument was an adaptation of the Center on Education and Work, at the 
University of Wisconsin, STW Self Assessment Check List. This instrument was designed for 
partnership program staff to meet as a group and evaluate their program by reaching a consensus 
based response to each question. The instrument was developed based on research of success 
indicators for these programs. The instrument is divided into seven categories, with sub- 
categories in each section. The seven main categories are: access, articulation, career 
development, collaborative partnerships, curriculum development, professional development, 
and work-based learning. Adaptation of the instrument was made with permission of the 
University of Wisconsin. This was done by changing words in some questions to directly reflect 
the automotive partnership program environment 

Data collection began with a site visit at each college. Each program coordinator 
provided a tour of their facilities, including training vehicles and shop equipment. During this 
visit, the self-assessment instrument was provided to the program coordinator. Copies were 
prepared for other instructors who teach in the automotive classes in the partnership prograin, the 
department dean, and counseling staff assigned to the partnership students. The instrument 
purpose was explained verbally and in a cover letter attached to each survey. Return envelopes 
were provided with each survey, along with a separate consent form and envelope to insure 
anonymity. 
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Based on the survey results, brief follow up interviews were conducted with the program 
coordinators for clarification of conflicting survey data. The self-assessment instrument was 
originally designed for use in a group setting. The group would discuss each question and 
respond based on consensus. This format was not used because of the time commitments it 
would impose on the staff at each site to meet and respond in a consensus manner to 146 
questions. 

Since each site had two or more surveys returned, the possibility for data conflicts 
existed. Conflicts were noted and then reviewed with the program coordinator at the respective 
site. It was felt that the coordinator was closest to the program and thus their response was used 
for these items solely. 

An appointment was made to interview the partnership program students during initial 
site visits and interviews were completed at each community college. Prior to commencing the 
interview, all students were read a consent form and given the opportunity to not participate. 

The interview consisted of eight questions as follows: 

1 . What can you tell me about the program coordinator’s leadership? (prompts: Are 
details of the program covered adequately? Is there genuine concern toward your success in the 
program? Is the coordinator available to discuss your problems or concerns?) 

2. What examples can you tell me about on the commitment of your instructors toward 
the program? (prompts: Are the instructors excited about the benefits of the program? Do they 
use the equipment that is donated? Are you able to apply what you learn on the job at the 
program work site?) 

3. Can you give me any examples that show how this program is committed to 
excellence? 

4. Do you regularly meet with the program coordinator, guidance counselor, and/or 
instructors to discuss career information and if so, describe how this may have helped you? 

5. What kind of discussions do you have with the work-site leader regarding career 
possibilities? 

6. Can you describe how you have matured since you began the program with regard to 
your abilities, knowledge, and self-confidence? 

7. What can you tell me about the non-automotive classes you took? (prompts: How will 
they help you in the future? Were they a waste of your time? Did you want more choices on 
what was required?) 

8. Now that you are close to completing this program and graduating, what are your 
plans for the future? 

The questions were designed to provided student responses regarding the success 
indicators of the school-to-work model. These included relevant experiences such as how the 
program coordinator provided guidance and support for them in the program. 

Results and Analysis of Data 



Data came fi'om four sources: the survey instrument, interviews with each program 
coordinator, student group interviews at each site, and program observation at each site. The 
primary data source was the survey. The survey was sub-divided into seven sections as follows: 
access, articulation, career development, collaborative partnerships, curriculum development, 
professional development, and work-based learning. 
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The results were reported for each section. The site observations and interviews 
enhanced the survey data using a different data collection method which provided opportunities 
for explanation and clarification. It should also be noted that the student interviews were critical 
to this research, as success for students completing the program ahd obtaining employment in 
automotive dealer service departments as technicians are why these programs exist. Therefore, 
student viewpoints were included to provide added additional data in this research. 

Data were first tabulated by site to create a master survey for each program. It was 
important to note that surveys were completed on an individual basis at each of the three sites. 
Responses to the same question could range from “fully implemented” to “not applicable,” 
dependent upon the perspective of the respondent. Several questions with opposing responses of 
“fully implemented” versus “not applicable” occurred at each site. A follow-up interview was 
made with the program coordinator at each site to clarify these conflicting responses in the 
survey data. They were chosen because they were the leaders of these programs at their colleges. 
The survey questions with conflicting responses demonstrated differing viewpoints on the 
program’s status among the respondents. There could have been many reasons for this, such as 
differing perspectives on the level of implementation from one person to another, differing 
interpretations of the survey question, lack of familiarity with an aspect of the program, and 
personal experience within the program, to name some. 

Data were then taken from each master survey to form a composite response of all three 
sites for each question. Data analyses were subsequently created from analyzing the composite 
responses, along with related site visit data, program coordinator interview data, and student 
interview data. The analyses were reported based on the seven section divisions of the survey 
instrument. 



Results 



The following are a reporting of research data from this study. It is reported by title of 
each section and applicable sub-sections of the survey instrument. Due to space limitations, the 
data tables are not included here but are available at the presentation. 

Access 

Recruitment andEnrollment 

Each site had printed materials they distributed to potential students. The materials 
included those provided by the automotive corporate sponsor, as well as those created by the 
community college automotive department. These materials were made available to prospective 
students at the college and various recruitment activities, such as high school career days, 
outreach to high school automotive programs, and student days at automotive events such as the 
National Hot Rod Association Winter Nationals (drag racing) in Pomona, California. It should 
be noted that outreach before the 1 1th grade has been identified as an irriportant element in the 
ST WO A, yet was only partially implemented at these sites. 
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Assessment 



Absent from all three programs was vocational and aptitude testing for all students. 

When a student had test scores, they were shared with the faculty. The T-TEN site did require 
each student to write a short paper on why they wished to enter the program as part of the 
enrollment process. The program coordinator stated that this helped eliminate students with 
marginal interest. If they could not take the initiative to write this paper, they may not have what 
it takes to get through the program. 

Support Services Related to School-to-Work 

All sites had dedicated counseling staff, though student’s opinions at each site varied. At 
the T-TEN site, 10 of the 23 students met with a guidance counselor, with one student stating 
“the instructors act as our counselor, and they end up being better than the guidance counselor.” 
Other students agreed with the statement. Another student added: “They [guidance counselors] 
don’t understand what we do here [in the program].” The situation was similar at the ASEP site, 
with many students never meeting with their assigned guidance counselor. In sharp contrast was 
the ASSET site, where all students met with their counselor. One student stated: “the best 
[guidance counselor] in the school; she understands this [ASSET] program.” Another stated: 
“She is smarter than most [counselors].” The program coordinator also stated that the counselor 
was very supportive and worked with him to insure students were on track to graduate on time. 

Classroom AccommodationsRelated to School-to-work 

Accommodating special populations students was implemented at each site. The ASEP 
coordinator spoke of the recent graduation of two deaf students. These students had sign 
language assistants present for course lectures. There was also mention of a graduate who used a 
wheelchair for mobility. The ASEP coordinator stated that course materials and activities were 
not specifically designed to accommodate special populations. When a student with special 
needs enrolled in the program, adjustments were made to accommodate the student. Each 
program coordinator mentioned that support staff were assigned when needed to insure 
accommodation of students with special needs. The ASSET coordinator stated he used English 
as a second language and math tutors for students with these needs each semester. 

Inclusive Approaches/Strateeies 

None of the sites had any activities related to inclusive strategies for special populations 
nor had they implemented strategies for proportional participation. All schools reported they had 
thorough documentation procedures for school to work activities for all students. 

Articulation 



Articulation Agreements 

All sites had articulation agreements with secondary schools and four year colleges. 
None of the sites listed competencies to be achieved in the articulated courses. All articulation 
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agreements listed courses for which credit would be given, yet the procedures for this were 
present at two of the three sites. With regard to articulation with high schools, the ASEP 
coordinator mentioned that if a student asked for articulation credit, they would grant it. There 
was no effort to encourage students to exercise this option. One coordinator stated he preferred 
that the student take the articulated class (at the college) so they could evaluate student 
performance and attitude. All coordinators stated that they worked with students who planned to 
continue their education at four year institutions so they received articulation credit. 

Transcripted Creditand Dual Credit 

All sites provide transcripted course information to students. Upon transfer, the student 
transcript reflects this course work and the assigned letter grade. 

Advance Standing Credit 

All sites provided advanced standing course credit information to their respondents. One 
site provided this information on the student transcript, while one responded "not applicable" at 
their college. The other site had a split response of "should plan" and "not applicable." 

Post-Secondary Linkages 

One site responded that articulated course work included competencies required in the 
post community college program(s). The other sites responded "not applicable." This was not 
the same as responding in the negative to this question. The survey participants may not have 
been completely aware of articulation agreement details. Competency identification was 
identified as fully implemented at one site, with the others responding "not applicable." Again, 
awareness of articulation agreement elements and the level of communication, if any, with the 
community college may have affected the response. 

Career Development 



Career Exploration 

Career exploration was integral to the curriculum and activities in the partnership 
programs. This was discussed during student interview sessions at each site. Students indicated 
their course work and work place experience was preparing them for technician positions at an 
auto dealership. However, they stated they had little or no discussion on future education and 
career options beyond their current objective— completion of the partnership program. In the 
follow-up interview, the program coordinators were asked about this. The ASEP coordinator 
response was similar to the others: “The dealership partners are investing in these students, 
providing work based learning, and they want them [students] to stick around a while. So, we 
don’t spend a lot of time beyond getting that job at the dealer.” The ASSET coordinator 
mentioned that he discusses opportunities such as working for the Ford regional office and 
teaching. 
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Curriculum Map andCareer Map 



The ASEP and ASSET sites utilized curriculum/ career maps to help guide students 
through the program course work. The T-TEN site responded “not applicable” for most of this 
section, with partial “should plan” responses that scored equal to “not applicable.” The ASEP 
and ASSET programs are very structured with regard to curriculum and sequence. As mentioned 
eairlier, the ASSET program divided each 18-week semester of the two-year program into a 9- 
week session at the college and 9-weeks at the dealer (work-based component). This format 
allowed the use of a very specific program for the ASSET student, but also reduced options in 
selecting elective courses. Any elective had to be part of the 9-week format or the student would 
have had to arrange his/her school, work, and personal schedule to attend an 1 8-week course. 

The T-TEN program was less structured and also allowed students more latitude in selection. 
This may account for the T-TEN responses of “not applicable” to these questions. 

Career Major 

The three partnership programs provided students with very strong career-oriented 
training. Except for linkages to four-year universities, all sites reported full implementation of 
the elements listed. By design, each program integrated the school-based learning with the 
work-based learning. For example, the ASSET coordinator noted that as students completed 
course work in transmissions, they would spend the next 9-weeks of the work-based learning 
component in the transmission department at the dealership. The other sites had similar 
arrangements. With so much time spent working in the dealer, the programs were structured to 
support this with adequate and relevant classroom activities. 

Career Plan 

Each program had work sheets that identified course work and sequencing for students. 
The students met with the coordinator every semester to review course scheduling for the next 
semester. The students keep a copy of the work sheet so they could reference it at any time. At 
each of these sites, most students progressed as a group through the program. This provided peer 
reinforcement as they shared in their experiences in course work and job experience. Only the 
ASEP site reported “not applicable” to a question in this section. The ASEP coordinator stated 
career planning was left at the auto dealer level rather than the college. The dealers who 
provided work-based learning environments, paying job positions, and hired the students as 
technicians after graduation, preferred to offer career growth opportunities once the student was 
working full tinie. 

Career Portfolio 



The responses to the career portfolio questions showed little to no implementation of 
these elements. During the follow-up interview with the program coordinators, each stated this 
was an area that needed exploration, and the data supported this from the number of "should 
plan" responses. At the time of the study, there were no requirements for students to create or 
maintain a portfolio. The ASSET coordinator mentioned that students received certificates of 
completion for course work and their grade reports. 
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Career Development 



Responses to questions relating to elements of career development activities at each site 
show only one site did this on a semester basis. All reported they had community member 
participation. The business and labor participation usually involved activities with dealer 
partners of each program. 

Job Shadowing 

The ASEP and ASSET programs reported full implementation in this area. The T-TEN 
program reported this area was not applicable. In discussing the results in this section with the 
coordinators, the term job shadowing caused confusion. The T-TEN site responded "not 
applicable" because the students were working, not shadowing, at the job site. All three of these 
programs rely on strong integration of school and work-based learning. Caution must be 
exercised in interpreting the data in this section due to different interpretations of the term “job 
shadowing” at these sites. 

Collaborative Partnerships 



School-to-Work Coordinator 



Each coordinator markets the program through distribution of print materials and visits to 
local high schools. These partnership programs have been on-going for several years, therefore 
many potential students had already known of the program. Each coordinator mentioned student 
recruitment had not been difficult, with some students seeking out the program on their own. 
Each student was responsible for obtaining placement in a dealer for work-based learning. The 
coordinator provided the student with a list of interested dealers and the student then interviewed 
with the dealer. Occasionally, the coordinator would intervene if a student was having difficulty 
finding placement. The coordinators spent a great deal of time recruiting and maintaining dealer 
relationships for the students. The ASSET coordinator’s statement was similar to the others: 
"Getting dealers to participate was sometimes difficult, though once they go through it and saw 
the benefits, they sought out students." 

Active Participation/Contribution 

Participation and contributions in these partnership programs was positive. Student 
comments at the ASSET site were typical of the responses of the three sites with regard to 
equipment, special (product specific) tools and vehicles: “We have all the Ford special tools, 
newest greatest equipment, and cars to work on.” The coordinators all indicated their business 
partners (dealers and corporate sponsor) provide essential tools, student materials, and vehicles at 
more than adequate levels. For example, the ASSET site had 22 vehicles donated by Ford, while 
the ASEP and T-TEN sites had similar numbers. 

Student materials for product specific courses were developed by the corporate sponsor. 
These materials were also used by dealer technicians when they attended corporate training 
courses. This system insured that the students received current and relevant information and 
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allowed the students to receive corporate credit for this course work in the corporate sponsor 
training system. This was valuable for the student, as corporate sponsors often certify dealer 
technicians, partly based on training completed. Upon graduation, the student was well on the 
way to achieving the "Master Technician" certification, which leads to higher pay and promotion 
potential. 

Collaborative Partners 



The partnership programs reported almost full implementation in relation to the questions 
regarding collaborative partnerships. By definition, these programs include business and 
industry leaders, both from the automotive corporate office and at the local level with dealer 
service personnel. The product specific curriculum was primarily developed by the business 
partner for use in corporate training classes and the partnership programs. All programs 
indicated full implementation in partner collaboration for recruiting organizations into the 
partnership. This was primarily corporate sponsors working with their dealers to get them to 
sign on and maintain their stake in the partnership. 

Strategic Planning 

Two of the three sites conducted annual meetings specific to the partnership. Of the two, 
only one reported setting goals and objectives specific to implementation of the program. Two 
sites reported using these meetings to plan coordinating activities among those involved. Only 
one site reported partial implementation for evaluating the effectiveness of the planning 
activities. 

Communication and Marketing 

The data show sporadic implementation in the area of conununication and marketing. 

One site responded “not applicable” to all questions in this area, except for partial 
implementation in education of all teachers and administrators about school-to-work. Another 
site reported full implementation in all areas except education of all teachers and administrators 
and regularly updating stakeholders about changes in school-to-work. The third site responded 
primarily in the planning and should plan areas. 

Curriculum Development 



Integrated and Applied Instruction 

All three sites reported full implementation of integrated and applied instruction. Each 
program utilized product specific training materials in their course work. This material included 
print, video, computer pro-grams, training simulators, and guided hands-on practice on vehicles 
and vehicle components. Use of these materials provided the students with relevant instruction 
and practice sessions that were directly applicable to their work-based experiences. These 
materials were the same ones their work place mentors used when they attended corporate 
training classes. Students at each site said they felt their course work had direct application to 
what they were doing in their work-based experiences. 
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Integrated and Applied Content 



Data again show high implementation of the elements in this area. The use of corporate 
training materials provided a solid base for product specific, up-to-date, automotive training, 
which combined with other automotive and academic courses gave students the knowledge to 
apply in the work-based environment. Student comments at each site were similar in this area, 
with T-TEN students stating their course work had integration and relevance in their hands-on 
experiences. The program curriculum maps guided the students through the courses in a 
recommended sequence. 

Integrated and Applied Assessment 

These programs have incorporated performance-based education into their curriculum. 
Again, the use of corporate training materials, including books, video, computer based learning, 
tools, and vehicles, allowed the instructor to evaluate students not only in the cognitive domain, 
but psychomotor and affective domains. Work was often done in groups in the college 
environment, due to the number of tools, components, or vehicles required for a specific activity. 

Professional Development 

The area of professional development overall showed limited implementation at these 
sites. It was important to note that at the ASSET site, there was one faculty member who 
coordinated the program and taught all the automotive courses for the program students. This 
made it difficult for the coordinator to participate in these activities if they conflicted with the 
semester schedule. As the ASSET coordinator stated: “If I’m not here, the program stops.” At 
the ASEP site, the coordinator went on sabbatical for one year. The college had to find a 
qualified individual to hire for one year in order to keep the program going. This was not easy, 
due the product specific nature of the course work and administrative duties of the program. 

Faculty Intemships/Job Shadowing Opportunities 

Data in this area show varied implementation at these sites. There was some 
participation in intern- ships and job shadowing from the survey data, and interview data with the 
coordinators confirmed that participation was sporadic at best. When faculty participate in these 
activities, the data show they are allowed to alter curriculum and have the opportunity to share 
their experiences with other faculty. 

Joint In-Service/Staff Development Opportunities 

All three sites reported conducting faculty in-services. Upon clarifying these data with 
the program coordinators, there was a common theme at each site. The in-services covered a 
variety of topics outside of the partnership program realm. There was no mentions of in-services 
specific to the partnership program by the coordinators. This, combined with the survey data, 
show there may be opportunity for conducting partnership program in-service activities. 
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Work-Based Learning 



The data in this area highlight work-based experiences for the students as a great strength 
of these programs. The student interviews confirmed this. The students at each site stated their 
work-based experiences complimented their course work and saw the application of it in the real 
world setting. It should be noted that the course work methods did not always match the 
application in the work place, with students at the ASSET site stating: "Here [in college] we 
learn the factory method, but in the dealer we learn the slammers [production] method." 

School Mentor/STW Coordinator Training " 

The data in this section show full implementation in the three pro-grams. Again, the 
strong support from the corporate sponsor insured that these programs provide up-to-date 
relevant experiences. The students recognized this. When asked why they would recommend 
their program to a potential automotive student, statements were made regarding the work-based 
experiences and the focused product training as the best ways to employment with the dealer. 
These students not only graduated with an Associate of Arts degree, they received corporate 
training credit, making them highly desirable in the job market in the eyes of the auto dealership. 

The data show no implementation at any of the sites for school mentor and program 
coordinator training. One site reported they should plan these activities. It was important to note 
that each program coordinator was questioned about the data in this area during the follow up 
interview. The three coordinators were similar, stating their programs did not use mentors at the 
college site. All three coordinators had formal education in the field of education, so the data, 
with respect to the coordinators, did not appear accurate in this section. This may be due to 
unfamiliar terminology (i.e., mentor, coordinator). One survey participant circled these terms on 
the instrument and wrote: “We do not use these terms.” Head instructor and program 
coordinator may be more common terminology in these programs. 

School Mentor/STW Coordinator 

The T-TEN site reported full implementation to each question in this category. The 
ASSET site reported partial to full implementation, and the ASEP data showed full 
implementation with the exception of identifying needs for additional learning opportunities. 
During the site visits, each coordinator stated they met regularly with each student and student 
comments at each site confirmed this. In these meetings, student progress was reviewed and the 
student had the opportunity to discuss his/her concerns. All coordinators stated they utilized 
campus resources for students who needed assistance, such as tutoring or referral to a counselor. 
The coordinators at each site stated they visited their dealer sites at least once a semester for 
feedback on student progress. 

Work Place Mentor Training 

The data received from each site was initially difficult to decipher in this area. This was 
due to conflicts in the responses from the individual sites. The data were often split between 
"full" or "partially implemented" and "should plan" or "not applicable." To overcome this, the 
questions in this area were reviewed with each program coordinator during the follow-up 
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interview. None of the sites conducted any formal training for the work place mentors. The 
ASEP program conducted workshops for the work place mentors to give them an overview of 
the program. The T-TEN and ASSET coordinators explained their programs to the participating 
dealer personnel on an individual basis. 

Work Place Mentor 



Data on work place mentors which show high levels of implementation and another area 
of strength in these three programs. The coordinators stated that the sponsoring dealers used 
their experienced technicians as mentors. This ensured the student was observing and working 
under a technician who was competent. The sites responded that their students were being 
challenged in the work place. The T-TEN and ASSET coordinators stated they discussed student 
standing with regard to current curriculum to help insure the students received hands-on practice 
relevant to the current classroom experience. 

The student comments at each site regarding their work place experiences were similar. 
As mentioned earlier, they felt the experience overall was invaluable. Regarding constructive 
feedback, an ASSET student commented: “I only hear if I make a mistake.” Others stated they 
met twice during their 9-week period at the dealer and others received non-verbal feedback: “If 
[the technicians] ask for our help, or if they avoid you, we know how we are doing.” The ASEP 
students stated they met with their mentor every day, and the T-TEN students once a week. 
Overall, the students said they were given work assignments or assisted the mentor with 
feedback most often when a mistake had been made. 

Conclusions 



The success of these programs was rooted in the commitment and dedication of the 
college faculty, combined with the strong support of the business partners. The program 
coordinators voiced strong beliefs and dedication to their partnership programs during the site 
visits. This was also evident from visual observation. Each site was extremely well organized, 
with large shop areas, well stocked tool rooms and cabinets, and 20 to 30 vehicles for hands-on 
training practice. The students were very focused on the activities; working on vehicles in 
groups, completing a hands-on performance test for assembling a transmission, asking relevant 
questions in class, and working individually to complete computer-based training. The student 
comments regarding their program coordinator, relevant program instructors, and college auto 
shop facilities were very positive at each site. 

Use of corporate training materials in the college course work ensured the students were 
receiving up-to-date, relevant education. These were the same materials their mentor technicians 
used during corporate training classes. The work-based learning component reinforced the 
classroom experience and provided real world work experience for the students. The students at 
each site stated their course work, combined with their work-based learning experience, prepared 
therri for a technician career. 

There were clear gaps toward full implementation in some indicator areas, including long 
range student career planning, integrating partnership activities into college strategic planning, 
professional development activities for faculty, and work place mentor training. While these 
programs are very career centered, they are focused on the immediate goal of employment as a 
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technician. Although it was mentioned as a politically sensitive issue by one program 
coordinator, long range career planning should be considered for inclusion in these programs. 

Communication and inclusion with other campus departments about the partnership 
program showed low levels of implementation from the survey data and student comments. The 
program coordinators at each site indicated a high work load and high level of responsibility 
associated with maintaining these programs. Two of the three sites did not rely on counseling 
staff support for their students. Efforts to improve counseling staff awareness of the program 
and needs of the students was oiie example of how improving communication could improve 
support for the program and the students. 

Professional development of the automotive partnership program faculty presented a 
challenge. Data showed low implementation in the area of planned growth activities for the 
faculty. These activities could include time spent working with work place mentors to keep 
current on vehicle repair techniques, attending educational and automotive training, and cross- 
training staff, such as with counselors, about the partnership program. Long term growdh 
activities at one site presented a challenge. The impending sabbatical leave of a program 
coordinator required finding a qualified individual to fill in for a one year period. There may be 
an opportunity here for industry to assist by giving one of their qualified employees a sabbatical 
by placing them at the college as a coordinator or instructor. The parties involved should spend 
time creatively exploring how to solve this problem. 

Work place mentor training was a critical issue at each site. Realistically, it would be 
difficult to expect mentor technicians to have formal educational training. However, it was 
reasonable to expect them to possess excellent communication skills, critique effectively, and use 
a college approved student assessment system to provide feedback to the students. The sites may 
continue in the spirit of self-assessment and look into each of these areas and develop action 
plans to strengthen them. 



Program Differences 

There were some differences among these three programs. A major difference was class 
scheduling in the ASSET program. This program operated on a different schedule than the 
standard 1 8-week semester of the college. They spend 9-weeks in class, then 9-weeks at the 
work site. The T-TEN and ASEP programs have students doing both school and work-based 
learning concurrently. The students in the ASSET program stated this schedule limited their 
general education choices as they did not cover all materials in the course syllabus and were 
taught by part-time faculty who they felt were not as caring or qualified as the 18-week 
instructors. Another difference was the curriculum structure of the T-TEN program. The ASEP 
and ASSET programs were very structured by the corporate sponsors. This provided consistency 
from college to college. The T-TEN program identifies minimums that students must meet, but 
leaves implementation details up to the college. Although this gave the college greater latitude 
for program structure, it also meant T-TEN students who attended different college sites received 
very different experiences. 
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Recommendations 



Insight gained from this research work resulted in the following recommendations for 
similar studies: 

1. Provide a glossary of terminology with the survey instrument. 

2. Explore the possibility of having the sites complete the survey as a group, reaching 
consensus for each item. 

3. Review the survey instrument with the participants to insure understanding of the 
intent of each section. 

4. When distributing the Self-Assessment instrument for individuals to respond to at the 
site(s), anticipate respondent answers that conflict. This will require follow-up by interviews or 
other means to obtain clarification of the data. 

Some surveys were returned with comments regarding the terminology used in the 
instrument, including "We don't use this terminology," and "I am unsure what is being asked." 
Also, in the follow-up interviews with the program coordinators, each mentioned they were not 
familiar with some of the terminology, causing confusion when responding to questions that 
included unfamiliar terms. Use of a glossary may have alleviated these problems. 

The surveys were distributed in sufficient quantities for all faculty and staff involved with 
the partnership at each site. This methodology gave the participants the flexibility to complete 
the instrument at a convenient time. This did provide necessary data for this research. However, 
not all faculty and staff at each site participated and because the surveys were completed 
individually, there were several survey questions at each site that had a response of fully 
implemented versus not applicable. The survey instrument design used response selections 
based on implementation level, planning level, or not applicable. Unlike a Likert scale, a fully 
implemented response and a not applicable response cannot be averaged out to equal a plaiming 
response. Response disparity was overcome by conducting follow-up interviews with the 
program coordinators for clarification on the questions that had conflicting responses. To help 
lessen this problem, the researcher could explore the possibility of having personnel at the 
research site(s) complete the survey in a group format. All stakeholders could participate and the 
response to each question would be in one of the five categories. It should be noted that the 
University of Wisconsin developed this survey instmment for use in this manner. However, this 
methodology could present challenges including: 

1 . Are the personnel at each site willing to commit the time resources necessary to 
complete a survey with 146 questions? 

2. Can the personnel easily schedule to meet together to complete the survey? 

3. Do they have the skills necessary to reach consensus on each question? 

Due to the challenges listed, this researcher distributed the surveys individually. The cost and 
benefits of these methodologies should be carefully considered by the future researcher. 

When the surveys were distributed to the program coordinators, the overall intent of the 
instrument was explained. A cover letter was also attached to each to insure the other 
respondents had the same information. Based on the feedback from the program coordinators, it 
would have been beneficial to review each section with each participant. This could have 
cleared up initial questions from the participants, including intent of the research, how to 
respond, and any questions on unfamiliar terminology. 
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Abstract 

The purpose of the research was to determine the stages of concern (SoC) of JobReady 
practitioners at the secondary level in North Carolina. The SoC dimension of the Concerns- 
Based Adoption Model (CBAM) was used to assess concerns of superintendents, vocational 
directors, principals, assistant principals, guidance counselors, academic teachers, and 
vocational teachers in the original 15 grant-funded JobReady partnerships. The SoC 
Questionnaire, with a section added for demographic and related information, was used to 
survey the education groups. 

Significant differences existed among JobReady partnerships based on configuration at 
the consequence, collaboration, and refocusing stages of concern. Single configured 
partnerships contained one public school system and multi-configured partnerships contained 
more than one public school system. Significant differences existed among JobReady 
partnerships when categorized by size at the awareness, informational, and collaboration SoC. 
Significant differences existed among the seven educator groups at all stages of concern. 
Significant positive relationships existed between perceived levels of JobReady involvement and 
highest SoC for principals, assistant principals, guidance counselors, academic teachers, and 
vocational teachers. 

JobReady partnership configuration and size impact on JobReady implementation. 
JobReady leaders should consider these characteristics when both establishing partnerships and 
allocating human and material resources. Superintendents, vocational directors, and principals 
indicated most intense concern at the consequence and collaboration levels. Assistant 
principals, guidance counselors, academic teachers, and vocational teachers expressed the most 
intense concern at the personal level. It was concluded that obtaining involvement of school- ~ 
based educators in JobReady strategies was an effective strategy for advancing SoC and 
facilitating implementation. 
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Introduction 



Concerns that students are leaving American high schools unprepared to succeed in the 
workplace have made school-to-work transition (STWT) a popular education reform topic. 
According to the Florida Coimcil on Vocational Education (1994), the United States may have 
the least effective STWT system of any industrialized nation. Glover and Marshall (1993) 
portrayed STWT as among the weakest links in the American learning system (p. 589). Others 
maintain that this coimtry does not even have a STWT system (Mendel, 1994). 

The STWT initiative has been referred to as both a renaissance (Beaumont, 1994) and a 
revolution (Brustein & Mahler, 1994, p. 22). Brustein and Mahler (1994) further maintained that 
the new emphasis on the development of a comprehensive STWT system has developed from 
changes in workforce needs and alarming facts about the preparation of American students for 
work and further education. 

Congress conveyed its vision of an effective school-to-work system through passage of 
the School-to-Work Opportunities Act of 1994. This act was hailed as a giant step toward the 
development of an educational system that matches students’ educational attainment and 
corresponding skills more closely to job opportimities (Brustein & Mahler, 1994). The act 
identified three basic STWT components as school-based learning, work-based learning, and 
connecting activities. This implication that the school and the workplace will be intertwined as 
never before carries with it new responsibilities for both businesspersons and educators. 

Significance and Statement of the Problem 

In March 1995, sixteen partnerships in North Carolina received initial planning grants to 
implement JobReady: Pathways to Career Success (hereafter, referred to as JobReady). 

JobReady is the state’s STWT system. North Carolina has since received a federal grant to 
further implement the system. Additional planning and implementation grants were awarded 
JobReady partnerships in March 1996. Funding for the JobReady initiative is expected to 
continue over the next several years. This study was needed to examine implementation concerns 
of early adopters so that future partnerships could benefit from the findings of this study. 

Reform initiatives such as JobReady provide practitioners an avenue to change 
expectations, attitudes, teaching strategies, and organizational structure. However, the benefits 
of educational innovations cannot be fully realized until these practitioners, including 
administrators, counselors, and teachers, adopt and implement change in the classroom (Long, 
1994). 

The levels of concern held by practitioners may influence the extent to which an initiative 
is implemented. Information is needed to determine the concerns of practitioners regarding 
implementation of JobReady strategies. State and local JobReady leaders need to know more 
about practitioners’ levels of concern with regard to the following: 

1. implications of JobReady for the practitioner; 

2. demands of JobReady on the individual; 

3. management, time, and logistical aspects of the innovation; 

4. impact of JobReady on students; 

5. interaction with others needed for JobReady implementation; and 
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6. potential for more imiversal benefits of the JobReady initiative. 

Effective and efficient change strategies to reach the successful reform level must be 
formulated. Attention must be given to the delivery of these strategies at the local level. 
JobReady leaders need insight as to how JobReady partnerships should be organized and funded 
to obtain maximum support from practitioners. The concerns of practitioners need to be 
addressed if JobReady is to have sustained impact. 

Purpose of the Study 

The purpose of this study is to determine the stages of concern (SoC) of JobReady 
initiative practitioners at the secondary level in North Carolina. The study was guided by the 
following research questions: 

1 . Is there a difference in the stages of concern of original grant-funded JobReady 
partnerships in North Carolina when categorized by either size or configuration? 

2. Is there a difference in the stages of concern of JobReady among the superintendents, 
vocational directors, principals, assistant principals, guidance counselors, academic 
teachers, and vocational teachers? 

3. Are the highest stages of concern for superintendents, vocational directors, 
principals, assistant principals, guidance counselors, academic teachers, and 
vocational teachers related to perceived levels of JobReady involvement? 

To address these research questions, the following null hypotheses were formulated: 

Hoi There is no statistically significant difference in stages of concern related to 
JobReady among original grant-funded partnerships with different forms of 
configuration. 

Ho2 There is no statistically significant difference in stages of concern related to 
JobReady among original grant-funded partnerships varying in size. 

Ho3 There is no statistically significant difference in stages of concern related to 
JobReady among superintendents, vocational directors, principals, assistant 
principals, guidance counselors, academic teachers, and vocational teachers. 

Ho4 There is no statistically significant relationship between highest stages of concern 
related to JobReady and perceived level of JobReady involvement among 
superintendents, vocational directors, principals, assistant principals, guidance 
counselors, academic teachers, and vocational teachers. 

Theoretical Framework 

The theoretical framework of this study is groimded in the body of knowledge referred to 
as change theory. This knowledge supports the premise that individuals adopt new initiatives 
transition through stages. The Concems-Based Adoption Model (CBAM) utilized in this 
research study was developed from this theoretical base. 

Curtis (1991) observed that “the difficulties of understanding and attempting to change 
human behavior have been great enough to be avoided by many brilliant minds attracted to 
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challenging problems” (p. 6). Fullan (1982) characterized educational change as being complex, 
difficult to define, and multidimensional. Katz and Kahn (1978) acknowledged that “attempts to 
change organizations by changing individuals have a long history of theoretical inadequacy and 
practical failure” (p. 658). Authorities concur that change is a process and acknowledge the 
importance of the individual in this process (Hall & Hord, 1987; Fullan, 1982). Hall and Hord 
further declared that accepting change as a process with a personal dimension was imperative to 
understanding change. 

The recognition of change as a process implies the existence of a time dimension. A 
frequently used approach to conceptualizing this time dimension has been to describe various 
phases occurring during the change process (Hall & Hord, 1987). Sisk (1969) reported that 
Massachusetts Institute of Technology psychologist Edgar H. Schein had developed an effective 
change model for use in the development of management personnel. This model included steps 
labeled as unfreezing, changing, and refreezing. Long (1994) described these three stages as 
creating motivation to change, developing new responses based on new information, and 
stabilizing and integrating the changes. 

Research supports the premise that three broad stages comprise the change process. 
These stages are described by Fullan (1982) as follows: 

Phase I — variously labeled initiation, mobilization, or adoption - consist of the 

process which leads up to and includes a decision to adopt or proceed with 
a change; 

Phase II — implementation or initial use (usually the first two or three years of use) 
involves the first experience of attempting to put an idea or program 
into practice; and 

Phase III — called continuation, incorporation, routinization, or institutionalization - 
refers to whether the change gets built in as an ongoing part of the system 
or disappears by way of a decision to discard or through attrition, (p. 39) 

Fullan expanded these phases by including outcome or results as the fourth phase of a 
“detailed and snarled” (p. 40) change process. According to Fullan, the process is complicated 
by (a) numerous factors operating at each stage, (b) the altering affect of events at one stage on 
decisions made at previous stages, (c) the scope of the proposed change and the source of 
initiation, and (d) the inability to precisely demarcate the time perspective. Fullan (1982) implied 
that change facilitators must deal with the complexities of people involved in implementing 
change. 

Curtis (1991) maintained that change theories must recognize that an individual’s 
behaviors, thoughts, and feelings can be altered either within an existing framework or when the 
entire organizational system changes. For the individual, the educational change may impact 
“their occupational identity, their sense of competence, and their self-concept” (Fullan, 1982, p. 
33). 

In a discussion of teacher change as it relates to the staff development process, 
Richardson (1994) cited several studies supporting the premise that teacher beliefs are related to 
classroom practices. Noting conflicting views on whether changes in classroom practice precede 
or follow changes in beliefs, Richardson conceived the beginning of the teacher-change process 
as being dependent on the types of changes and the teachers themselves. Therefore, the change 
process could begin with either changes in beliefs or changes in practice. 
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In an examination of individual change as it occurs in work settings, Bunker and Delisle 
(1991) generalized that participation in the change process could facilitate change and lead to 
greater internalization. They identified two issues pervading all frameworks of organizational 
change efforts. The first issue identified targeted levels of individual change. The most internal 
levels, consisting of feelings and values, are more difficult to change than more external levels 
identified as skills and observable behaviors. The second issue related to the extent to which the 
individual internally accepts and integrates the change. If the change is to be sustained over a 
long period of time, it must be accepted and internalized by the individual so that it becomes part 
of their value system. 

Bunker and Delisle (1991) maintained that the change process begins for an individual or 
group when a gap is perceived between actual experiences and desired experiences. They also 
reported that people commonly resist change because of impact on their values, a lack of 
understanding about the change and its implications, a belief that the change is irrelevant, and a 
low personal tolerance for change. Hall and George (1979), citing studies that document the 
complexity of educational change, noted the important role of teachers in the process. They 
concurred that innovations fail due to inadequate consideration of the human element. For 
educational endeavors to be successful, teachers’ reactions to the program must be taken into 
account. 

The CBAM was developed by the University of Texas Research and Development Center 
(UTR&DC) to conceptualize and facilitate educational change (Long, 1994). Hall and Hord 
(1987) noted consistent results in studies of teachers’ problems and satisfactions, which indicated 
the existence of change patterns. “These early studies and data became the first indicators that 
the differing perceptions and needs of teachers are important considerations when developing 
and delivering teacher interventions” (p. 55). This research base indicated that different 
teachers’ experiences should be considered in implementing classroom change. 

In working with individuals involved in change, staff at the UTR&DC found concerns 
about change to be an important dimension of the process. In their research, “innovation” was 
the generic name given to the issue, object, problem, or challenge, or the thing that is the focus of 
the concerns. The innovation and its use provide a frame of reference from which concerns can 
be viewed and described (Hall, George, & Rutherford, 1977). 

Hall and Hord (1987) described “concerns” as a phenomenon confronting everyone faced 
with new experiences, demands for improvement, and changes. “Concerns theory applies to 
teachers, principals, teacher educators, and others from a multiplicity of tasks and roles” (p. 58). 
Their concept of concerns is described as follows: 

The composite representation of the feelings, preoccupation, 
thought, and consideration given to a particular issue or task is 
called concern. Depending on our personal make-up, knowledge, 
and experience, each person perceives and mentally contends with 
a given issue differently; thus there are different kinds of concerns. 

The issue may be interpreted as an outside threat to one’s well 
being, or it may be seen as rewarding. There may be an 
overwhelming feeling of confusion and lack of information about 
what “it” is. There may be ruminations about the effects. The 
demand to consider the issue may be self-imposed in the form of a 
goal or objective that we wish to reach, or the pressure that results 
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in increased attention to the issue may be external. In response to 
the demand, our minds explore ways, means, potential barriers, 
possible actions, risk, and rewards in relation to the demand. All 
in all, the mental activity composed of questioning, analyzing, and 
re-analyzing, considering alternative actions and reactions, and 
anticipating consequences is concern. An aroused state of personal 
feelings and thought about a demand as it is perceived is concern. 

(p. 58-59) 

Fuller identified four major concern clusters as unrelated, self, task, and impact. 
According to Hall and Hord, these clusters were verified through the research of Fuller and 
others. Although all concerns focused on teaching, each cluster possessed a different content and 
dynamic. Hall, Wallace, and Dossette (1973) found that Fuller’s developmental concerns 
concept could be applied to the process of innovation adoption. Long (1994) reported that 
Fuller’s work “provided the basis for developing the Stages of Concern About the Innovation 
dimension of the Concems-Based Adoption Model” (p. 33). 

The research base indicated the existence of “a set of developmental stages and levels 
teachers and others moved through as they become increasingly sophisticated and skilled in 
using new programs and procedures” (Hall & Hord, 1987). While recognizing other variables of 
the change process, the CBAM position is that the primary focus must be upon the individual 
iimovation user. Other user system variables can be examined in relation to how they affect the 
concerns of each of the individuals (Hall, 1975). 

The Stages of Concern About the Iimovation dimension of the CBAM investigates the 
concerns that users have about an innovation from limited awareness until mastery. The seven 
stages of concern (SoC) displayed in Figure 1 are identified as Stage O - Awareness; Stage 1 - 
Informational; Stage 2 - Personal; Stage 3 - Management; Stage 4 - Consequence; Stage 5 - 
Collaboration; and Stage 6 - Refocusing. Hall and Hord (1987) stated that these SoC could be 
grouped into self, task, and impact dimensions. Long (1994) reported that self-concerns include 
Stages 0, 1, and 2. This phenomena occurs early in the change process as individuals desire to 
know more about the innovation. Task-concerns are synonymous with Stage 3 and are 
associated with the individual becoming more intense during the early period of innovation 
use. Impact-concerns incorporate Stages 4, 5, and 6 and relate to the individual’s attention given 
to impact on students and improving effectiveness of the innovation. 

Figure 1 : Stages of Concern about the Innovation 



0 Awareness : Little concern about or involvement with the innovation is indicated. 

1 Informational : A general awareness of the innovation and interest in learning more detail 
about it is indicated. She/he is interested in substantive aspects of the innovation in a 
selfless manner such as general characteristics, effects, and requirements for use. 

2 Personal : Individual is uncertain about the demands of the innovation, his/her inadequacy 
to meet those demands, and his/her role with the innovation. This includes analysis of 
his/her role in relation to the reward structure of the organization, decision making and 
consideration of potential conflicts with existing structures or personal commitment. 



Financial or status implications of the program for self and colleagues may also be 
reflected. 

3 Management : Attention is focused on the processes and tasks of using the innovation and 
the best use of information and resources. Issues related to efficiency, organizing, 
managing, scheduling, and time demands are utmost. 

4 Consequence : Attention focuses on impact of the innovation on students in his/her 
immediate sphere of influence. The focus is on relevance of the innovation for students, 
evaluation of student outcomes, including performance and competencies, and changes 
needed to increase student outcomes. 

5 Collaboration : The focus is on coordination and cooperation with others regarding use of 
the innovation. 

6 Refocusing : The focus is on exploration of more universal benefits from the innovation, 
including the possibility of major changes or replacement with a more powerful 
alternative. Individual has definite ideas about alternatives to the proposed or existing 
form of the innovation. 



Original concept from Hall, G. E., Wallace, R. C., Jr., & Dossett, W. A. A 
developmental conceptualization of the adoption process within educational institutions. Austin: 
Research and Development Center for Teacher Education, The University of Texas, 1973. 

Long (1994) reported that CBAM assumptions about change were verified through 
research. These assumptions are identified as follows: 

1 . Change is a process, not an event. Change is a process occurring over time, and recognizing 
this is an essential prerequisite of successful change implementation. 

2. Change is accomplished by individuals. Change affects people, and their role in the process 
is important. Therefore, individuals must be the focus of attention in implementing a new 
program. 

3. Change is a highly personal experience. Individuals are different and do not behave 
collectively. Each individual reacts differently to change and some will assimilate a new 
practice more rapidly than others. Change is more successful when its support is geared to 
the diagnosed needs of the individual users. If change is highly personal, then different 
responses and interventions are required. By paying attention to each individual’s progress 
one can enhance the implementation process. 

4. Change involves developmental growth. Studies indicate that the individuals involved 
appear to express or demonstrate growth in terms of their feelings and skills. These tend to 
shift with respect of the new program or practice as individuals pass through a greater degree 
of experience. 

5. Change is best understood in operational terms. Teachers, and others, will naturally relate to 
change or improvement in terms of what it will mean to them or how it will affect their 
current classroom practice. Change facilitators can reduce resistance to improvement efforts 
by addressing questions and communicating with teachers and others involved. 



6. The focus of facilitation should be on individuals, innovations, and the context. It is easy to 
forget that books, materials, equipment, or new programs alone do not make change; only 
people can make change by altering their behavior. The real meaning of change lies in its 
human, not its material component. 

Long examined several change studies using the CBAM. She noted that “the majority of 
studies used to verify the CBAM were in educational settings with iimovations of instruction, 
classroom configuration, and instructional administration” (p. 37). Long concluded that the 
CBAM provided an effective approach of determining individual differences in implementing 
iimovations. 

Research Methods and Procedures 

The population for this study consisted of educators in 1 5 original grant-funded JobReady 
partnerships in North Carolina. Sampling procedures were designed to ensure adequate 
representation of all educator groups and 15 partnerships. The total sample consisted of 1200 
individuals including the following groups: superintendents (N=25), vocational directors (N=36) 
principals (N=127), assistant principals (N=307), guidance counselors (N=457), academic 
teachers (N=6567) and vocational teachers (N=1482). The usable return rate for each group was 
superintendents (80.0%); vocational directors (88.9%); principals (72.2%) assistant principals 
(70.6%); guidance coimselors (68.3%); academic teachers (52.9%), and vocational teachers 
(65.9%). 

The SoC Questionnaire is a dimension of the Concems-Based Adoption Model (CBAM) 
which describes the feelings, perceptions, and attitudes of individuals as they consider, approach, 
and implement an innovation. The CBAM was used by Long in a Virginia study investigating 
levels of concern when implementing Tech Prep programs (1994). 

The SoC Questionnaire consists of 35 statements regarding the seven Stages of Concern. 
Five statements are related to each Stage of Concern. On a 0 to 7 Likert type scale, respondents 
indicate the degree to which each concern is true of them. For individuals or groups, computing 
mean scores for each stage to determine profiles summarizes the data. The higher the mean 
score, the more intense the concerns at that stage. The SoC Questionnaire also included nine 
demographic and other informational items on the reverse side. A comment sheet was included 
with the SoC Questionnaire for participants to share concerns or suggestions based on their 
experience with JobReady. 

The North Carolina instrument was reviewed by a panel of experts and subjected to a 
field test in order to establish content validity. To determine internal consistency, a correlational 
analysis of the 35 SoC Questionnaire items was conducted. The analysis yielded a Cronbach 
Coefficient Alpha of .90 indicating a high degree of internal consistency. 

Individuals in the sample were mailed the SoC Questionnaire packet. An over all 
response rate of 64.6% or 775 usable questionnaires was achieved after two mail and one phone 
call follow-up. There were no significant differences between early respondents and late 
respondents. 



O 

ERIC 



75 



86 



Findings 



The research questions were stated as null hypotheses. Analysis of variance (ANOVA) 
procedures indicated significant differences between single and multi configured partnerships at 
the consequence, collaboration, and refocusing stages (p < .05). Therefore, Hoi was rejected. A 
multivariate analysis of variance (MANOVA) procedure indicated a significant difference 
among partnerships when categorized as small, medium, large, and extra large . Therefore, H 02 
was rejected. Further analysis (ANOVA) indicated significant differences at the awareness, 
informational, and collaboration SoC. The Tukey-Kramer post hoc procedure did not detect 
differences at the awareness and collaboration stages. 

Null hypothesis 3 sought to determine if a statistically significant difference existed 
among the superintendents, vocational directors, principals, assistant principals, guidance 
counselors, academic teachers, and vocational teachers regarding the SoC of JobReady. A 
significant difference was indicated by a MANOVA procedure and further analysis (ANOVA) 
indicated significant differences at each stage of concern (the Wilks’ lambda statistic (.6072) was 
[F (42,3577) = 9.558, p = .0001]. Therefore, H 03 was rejected. The Tukey-Kramer post hoc 
procedure specified group differences at each concern stage. These differences are summarized 
in Tables 1-7. 

Null hypothesis 4 sought to determine if highest stages of concern among educator 
groups were related to perceived level of involvement. Correlation analyses were conducted for 
the total sample and for each group. The Kendall’s Tau b statistic (Kendall’s Tau b = .4048, p = 
.0001) revealed a significant positive relationship for the total sample. Therefore, H 04 was 
rejected. Positive relationships were indicated for principals, assistant principals, guidance 
counselors, academic teachers, and vocational teachers. 
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Table 1. JobReady Stages of Concern Profile for Superintendents 



Profile 


M 


Similar Groups 








Stage 0 - Awareness 


6.55* * 


— 


VD 


PR 


— 


— 


— 


Stage 3 — Management 


8.40* 


— 


— 




— 


— 


“ 


Stage 6 - Refocusing 


12.30 


— 


VD 


PR 


AP 


GC - 


VT 


Stage 2 - Personal 


14.60* 


— 


VD 


~ 


— 


— 


— 


Stage 1 - Informational 


16.65 


— 


~ 




— 


AP 


“ 


Stage 4 - Consequence 


24.60** 


— 


VD 


PR 


— 


— 


VT 


Stage 5 - Collaboration 


24.80 


-- 


VD 


PR 


-- 


-- 


VT 



Implications 

1 . Indicate little need for additional orientation 

2. Express less concern about management aspects than all other groups 

3. Are relatively comfortable with their role in implementation 

4. Are focusing attention on student impact and coordinating with others 



Note. Similar groups are based on Tukey-Kramer post hoc analysis 

* Lowest mean score of all educator groups 

**Highest mean score of all educator groups 
VD - Vocational Directors, PR - Principals, AP - Assistant Principals, 

GC - Guidance Counselors, AT - Academic Teachers, and VT - Vocational Teachers 
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Table 2. JobReady Stages of Concern Profile for Vocational Directors 



Profile M Similar Groups 



GC - 

AP GC AT 
AP GC - 



Implications 

1 . Indicate little need for additional orientation 

2. Express less need for information than all other groups 

3. Are relatively comfortable with their role in implementation 

4. Are focusing attention on student impact and coordinating with others 



Note. Similar groups are based on Tukey-Kramer post hoc analysis 
* Lowest mean score of all educator groups 
** Highest mean score of all educator groups 

SU - Superintendents, PR - Principals, AP - Assistant Principals, 

GC - Guidance Counselors, AT - Academic Teachers, and VT - Vocational Teachers 



Stage 0 - Awareness 


6.56 


SU - 


PR 


Stage 1 - Informational 


12.16* 





— 


Stage 6 - Refocusing 


13.88** 


SU 


PR 


Stage 3 - Management 


15.69** 


— 


PR 


Stage 2 - Personal 


16.06 


SU - 


— 


Stage 4 - Consequence 


22.84 


SU - 


PR 


Stage 5 - Collaboration 


25.94** 


SU 


PR 
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Table 3. JobReady Stages of Concern Profile for Principals 



Profile M Similar Groups 



Stage 0 - Awareness 


9.64 


SU 


VD - 


“ 


— 


— 


“ 


Stage 6 - Refocusing 


11.80 


SU 


VD - 


AP 


GC 


— 


VT 


Stage 3 - Management 


12.79 


— 


VD - 


AP 


GC 


AT 


VT 


Stage 1 - Informational 


20.89 


— 


— 


“ 


AP 


GC 


AT 


Stage 2 - Personal 


20.99 


— 


— 


“ 


AP 


GC 


AT 


Stage 4 - Consequence 


21.30 


SU 


VD - 


AP 


GC 


— 




Stage 5 - Collaboration 


23.11 


SU 


— 


— 


AP 


GC 


“ 



Implications 

1 . Indicate little need for additional orientation 

2. Are not considering alternatives 

3. Are focusing attention on student impact and coordinating with others 



Note. Similar groups are based on Tukey-Kramer post hoc analysis 

SU - Superintendents, VD - Vocational Directors, AP - Assistant Principals, 

GC - Guidance Counselors, AT - Academic Teachers, and VT - Vocational Teachers 



Table 4. JobReady Stages of Concern Profile for Assistant Principals 



Profile M Similar Groups 



Stage 6 - Refocusing 


9.87 


SU 


— 




PR 


— 


GC 


AT 


Stage 0 - Awareness 


12.20 


— 


— • 




PR 


— 


GC 


— 


Stage 3 - Management 


13.56 


— 


VD 


PR 


“ 


GC 


AT 


VT 


Stage 5 - Collaboration 


18.68 


— 


— 




PR 


— 


GC 


— 


Stage 4 - Consequence 


19.51 


— 


VD 


PR 


— 


GC 


AT 


VT 


Stage 1 - Informational 


19.74 


SU 


— 




PR 


— 


GC 


AT 


Stage 2 - Personal 


21.24 


— 


— 




PR 


— 


GC 


AT 



Implications 

1. Are not considering alternatives 

2. Indicate little need for additional orientation 

3. Have management concerns 

4. Indicate need for more information 

5. Are concerned about their role in meeting demands 



Note, Profile is same as for guidance counselors 

Similar groups are based on Tukey-Kramer post hoc analysis 

SU - Superintendents, VD - Vocational Directors, PR - Principals, 

GC - Guidance Counselors, AT - Academic Teachers, and VT - Vocational Teachers 
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Table 5. JobReady Stages of Concern Profile for Guidance Counselors 



Profile 


M 


Similar Groups 










Stage 6 - Refocusing 


10.37 


SU 


VD PR 


AP 


— 


AT 


VT 


Stage 0 - Awareness 


13.67 




— 


— 


AP 


— 


AT 


Stage 3 - Management 


15.68 


— 


VD PR 


AP 


— 


AT 


VT 


Stage 5 - Collaboration 


18.57 




“ 


PR 


AP 


— 


— 


Stage 4 - Consequence 


19.39 




VD PR 


AP 


~ 


AT 


VT 


Stage 1 - Informational 


21.30 


— 


— 


PR 


AP 


“ 


AT 


Stage 2 - Personal 


22.72 


— 


— 


PR 


AP 


-- 


AT 


Implications 
















1 . Are not considering alternatives 

2. Indicate little need for additional orientation 











3. Have management concerns 

4. Indicate need for more information 

5. Are concerned about their role in meeting demands 



Note. Profile is same as for assistant principals 

Similar groups are based on Tukey-Kramer post hoc analysis 

SU - Superintendents, VD - Vocational Directors, PR - Principals, AP - Assistant Principals, AT 
- Academic Teachers, and VT - Vocational Teachers 
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Table 6. JobReady Stages of Concern Profile for Academic Teachers 



Profile 


M 


Similar Groups 








Stage 6 - Refocusing 


1 . 11 * 


-- 


— 


AP 


GC - 


Stage 5 - Collaboration 


13.68* 


"" 


— 


— 


.. 


Stage 3 - Management 


14.21 


VD PR 


AP 


GC 


VT 


Stage 4 - Consequence 


16.27* 


— 


— 


AP 


GC - 


Stage 0 - Awareness 


16.92** 


-- 


— 


— 


GC - 


Stage 1 - Informational 


20.78 


■ „ 


PR 


AP 


GC - 


Stage 2 - Personal 


21.42 


— 


PR 


AP 


GC - 


Implications 












1 . Are not considering alternatives 

2. Are not collaborating with others 

3. Indicate need for more information 

4. Are concerned about their role in meeting demands 









Note. Similar groups are based on Tukey-Kramer post hoc analysis 
*Lowest mean score of all educator groups 
**Highest mean score of all educator groups 

SU - Superintendents, VD - Vocational Directors, PR - Principals, AP - Assistant Principals, GC 
- Guidance Counselors, and VT - Vocational Teachers 
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Table 7. JobReady Stages of Concern for Vocational Teachers 



Profile M Similar Groups 



Stage 0 - Awareness 


11.45 


-- 


— 




PR 


AP 


GC 


Stage 6 - Refocusing 


12.51 


SU 


VD 


PR 


AP 


GC 


— 


Stage 3 - Management 


15.68 


“ 


VD 


PR 


AP 


GC 


AT 


Stage 5 - Collaboration 


20.13 


SU 


— 




PR 


AP 


GC 


Stage 4 - Consequence 


21.53 


SU 


VD 


PR 


AP 


GC 


— 


Stage 1 - Informational 


22.07* ** 


— 


AT 




PR 


AP 


GC 


Stage 2 - Personal 


24.71** 


— 


AT 




PR 


AP 


GC 



Implications 

1 . Indicate little need for additional orientation 

2. Are not considering alternatives 

3 . Indicate need for more information 

4. Are concerned about their role in meeting demands 



Note. Similar groups are based on Tukey-Kramer post hoc analysis 

**Highest mean score of all educator groups 

SU - Superintendents, VD - Vocational Directors, PR - Principals, AP - Assistant Principals, 
GC - Guidance Counselors, and AT - Academic Teachers 



Conclusions 



Partnership configuration may have an effect on JobReady SoC. Examination revealed 
significant differences between single and multi configured partnerships at the higher concern 
stages of consequence, collaboration, and refocusing. Therefore, practitioners in single 
configured partnerships have a higher level of concern regarding JobReady impact on students 
and how JobReady might be improved. 

Partnership size may have an effect on JobReady SoC. Examination revealed significant 
differences at the awareness, informational, and collaboration stages. Based on their similar high 
mean scores recorded at the informational level, it is concluded that practitioners in extra large 
and large partnerships have greater needs for more information regarding JobReady than those in 
small partnerships. 

Testing of Ho3 found that educator group may have an effect on stage of concern at each 
level. At the awareness level, the highest mean score recorded by academic teachers was 
significantly different from all groups except guidance counselors. The lowest mean score 
recorded by superintendents was significantly different from all groups except vocational 
directors and principals. At the awareness level, little concern about or involvement with the 
innovation is indicated. Based on similar high mean scores recorded at the awareness level, it is 
concluded that academic teachers and guidance counselors need relatively more introduction and 
orientation to JobReady. Principals, vocational directors, and superintendents have less need for 
additional orientation. 

At the informational level, similar higher mean scores recorded by vocational teachers, 
guidance counselors, principals, and academic teachers differed significantly from lower mean 
recorded by superintendents and vocational directors. At the informational level, the individual 
displays a general awareness of the innovation and is interested in learning more about it. Based 
on the comparison of mean scores, it is concluded that vocational teachers, guidance counselors, 
principals, and academic teachers have relatively more need for information regarding JobReady 
than vocational directors and superintendents. 

At the personal level, the similar lower mean scores of vocational teachers, guidance 
counselors, academic teachers, assistant principals, and principals were significantly different 
from similar higher mean scores recorded by vocational directors and superintendents. At the 
personal level, the individual is unsure about the demands of the irmovation and his or her role in 
it. Based on the comparison of mean scores, it is concluded that vocational teachers, guidance 
counselors, academic teachers, assistant principals, and principals have relatively more concern 
for their role in meeting JobReady demands than superintendents and vocational directors. 

At the management level, the higher mean scores recorded by vocational directors, guidance 
counselors, vocational teachers, academic teachers, assistant principals, and principals were not 
significantly different. The lowest mean score recorded by superintendents was significantly 
different from all other groups. At the management level, attention is focused on the processes 
and tasks of using the innovation. Based on the comparison of mean scores, it is concluded that 
vocational directors, guidance counselors, vocational teachers, academic teachers, assistant 
principals, and principals have similar high levels of concern regarding processes and 
implementation tasks pertaining to JobReady. Superintendents are less concerned about 
JobReady processes and tasks than all other groups. 
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At the consequence level, the higher mean scores recorded by superintendents, vocational 
directors, principals, vocational teachers, and principals were not significantly different. The 
lowest mean score recorded by academic teachers was similar to scores recorded by guidance 
counselors and assistant principals. At the consequence level, attention is focused on the impact 
of the innovation for students, evaluation of student outcomes, and changes needed to increase 
student outcomes. Based on the comparison of mean scores, it is concluded that superintendents, 
vocational directors, principals, vocational teachers, and principals have similar high levels of 
concern regarding the impact of JobReady on students. Academic teachers, guidance counselors, 
and assistant principals have less concern regarding the impact of JobReady on students than 
other groups. 

At the collaboration level, the higher mean scores recorded by vocational directors, 
superintendents, and principals were similar. The lowest mean score obtained by academic 
teachers differed significantly from all other groups. At the collaboration level the focus is on 
coordination and cooperation with others regarding use of the innovation. Based on the 
comparison of mean scores, it is concluded that vocational directors, superintendents, and 
principals have similar high concern levels regarding cooperation and coordination with others 
regarding the use of JobReady. Academic teachers have less concern regarding cooperation and 
coordination with others pertaining to the use of JobReady than superintendents, vocational 
directors, principals, assistant principals, guidance counselors, and vocational teachers. 

At the refocusing level, the higher mean scores of vocational directors, vocational 
teachers, superintendents, principals, and guidance counselors were similar. The lower mean 
scores of academic teachers, assistant principals, and guidance counselors were similar. At the 
refocusing level, the emphasis is on obtaining more universal benefits of the innovation. Based 
on the comparison of mean scores, it is concluded that vocational directors, vocational teachers, 
superintendents, principals, and guidance counselors focus relatively more attention on 
exploration of more universal benefits from JobReady than other groups. 

Testing of Ho4 found a significant relationship between respondents’ highest stages of 
concern and perceived levels of involvement for principals, assistant principals, guidance 
counselors, academic teachers, and vocational teachers. A significant relationship was not found 
for either superintendents or vocational directors. Based on this finding it is concluded that 
getting school-based educators involved in JobReady components is a means of advancing 
concern levels and facilitating implementation. 

Discussion 

It is likely that more complicated partnership characteristics will impact JobReady 
leaders’ ability to fully implement JobReady. At the highest concern levels of consequence, 
collaboration, and refocusing, single configured partnerships recorded significantly higher mean 
scores than multi configured partnerships. These three stages collectively can be categorized as 
the impact level of concern. When categorized by size, extra large partnerships indicated a 
greater need of practitioners for more information concerning JobReady than small partnerships. 
JobReady leaders at the state and local levels must devise strategies to overcome coordination 
and information dissemination problems that are magnified as partnership size increases or when 
more than one school system is included. This finding is consistent with Fullan’s (1982) work 
indicating that the change process is complicated by the scope of the change and numerous 
factors operating at each stage of change. 
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The findings of this study indicated a significant positive relationship between . 
practitioners’ perceived levels of JobReady involvement and highest SoC. This phenomenon is 
consistent with Long’s (1994) finding of a positive relationship between educators’ Tech Prep 
involvement and highest SoC. The significant positive relationships support the research of 
Bunker and Delisle (1991) indicating that participation in innovation related strategies could 
facilitate change. 

When relationships of perceived JobReady involvement levels with highest SoC were 
considered for each educator group, all school-based personnel indicated a significant, positive 
relationship. No significant relationship was indicated for central office administrators 
(superintendents or vocational directors). The clear implication is that JobReady leaders will 
have to get school-based educators involved in related components. However, the majority of 
respondents in this study confessed no increased time spent in JobReady components and no 
involvement in JobReady professional development activities. Unless the involvement and 
support of these key educator groups are obtained, JobReady will have little chance of acquiring 
the prestige, mainstream status, or other essential reform characteristics described by Elford 
(1993). 

Sarason (1995) implied that the first target of educational change must be the attitudes 
and conceptions of educators. The seven educator groups included in this study must be key 
change facilitators for successful implementation of JobReady. Educator group was found to 
have a significant effect at every level of concern. The comparison of educator group mean 
scores by stage of concern indicates differences among groups and merits further examination. 

The SoC profile for superintendents (Table 1) indicates little concern for additional 
orientation to JobReady or issues related to efficiency, organization, and management demands 
In fact, superintendents indicated less intense concern at awareness and management SoC than 
all other groups. The superintendents’ lowest mean score of all groups at the personal level was 
similar only to vocational directors. This indicates that superintendents are more comfortable 
with their role in implementing JobReady than school-based educators. The highest mean score 
of all groups at the consequence stage of concern and second highest mean score at the 
collaboration level imply that superintendents are focusing attention on JobReady student impact 
and working with others regarding the initiative. Fullan (1982) identified the superintendents as 
the most important individual for establishing change patterns in high school districts. 
Superintendents must translate their knowledge of and high level concerns regarding JobReady 
into leadership for obtaining involvement and support of school-based practitioners. 

The SoC profile for vocational directors (Table 2) indicates little need for additional 
orientation to JobReady. Less need for additional information is expressed than for all other 
groups. Vocational directors indicated more intense concern for management aspects of 
JobReady than all other groups. The mean score of vocational directors at the personal stage was 
significantly lower than all school-based educator groups. This lack of intense concern at the 
personal level demonstrates that vocational directors are comfortable with their role in 
implementing JobReady. The second highest mean score of all groups at the consequence stage 
of concern and highest mean score at the collaboration level imply that vocational directors are 
focusing attention on JobReady student impact and working with others regarding the initiative. 
Fullan (1982) noted the leadership role of central office administrators in implementing and 
managing change. Similar to superintendents, vocational directors must fulfill this leadership 
role by translating their knowledge of and high level concerns regarding JobReady into strategies 
for obtaining involvement and support of school-based practitioners. 
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The SoC profile for principals (Table 3) indicates relatively little need for further 
introduction to the JobReady concept. Although not ready to consider alternatives to JobReady 
and similar to superintendents and vocational directors, they are mainly concerned about the 
impact of JobReady on students and working with others regarding the use of JobReady. As the 
school leader, the principal must translate his or her higher level SoC into a school organization 
and climate conducive to JobReady implementation. 

The similar profiles for assistant principals (Table 4) and guidance counselors (Table 5) 
indicate relatively little concern about JobReady alternatives or need for further introduction to 
the concept. Unlike higher level administrators, these groups indicate need for more JobReady 
information and show high concern about their role in meeting JobReady demands. Hall and 
Hord (1987) commented on the role of assistant principals as change facilitators. Hoyt et al. 
(1972) recognized the importance of guidance in implementing career-oriented strategies. 
JobReady leaders and the higher level administrators of this study must clarify the role of 
assistant principals and guidance counselors in implementing JobReady. 

The SoC profile for academic teachers indicates relatively little concern for either 
JobReady alternatives or collaboration with others regarding the concept. Most intense concern 
was demonstrated at the awareness, informational, and personal levels that are collectively 
described as self-concerns. The fact that academic teachers recorded the lowest mean scores of 
all groups at refocusing, collaboration, and consequence stages and the highest mean score at 
informational stage imply that JobReady has not been either adequately communicated to or 
accepted by academic teachers as a reform effort. Richardson (1994) associated teachers’ 
benefits with classroom practice. JobReady leaders must develop strategies designed to 
incorporate JobReady concepts into the value systems of academic teachers. 

The SoC profile for vocational teachers indicates little need for further orientation. 
However, vocational teachers indicated more intense concern for obtaining JobReady 
information than all other groups. Vocational teachers also expressed more concern about the 
personal demands of JobReady and their role in it than all other groups. JobReady leaders and 
higher level administrators must clarify the roles of vocational teachers in implementing 
JobReady. 

The status of teachers in the implementation of JobReady warrants additional comments. 
Hall and George (1979) noted the important role of teachers in the complex educational change 
process. Fullan (1982) found educational change at the classroom level manifested in the use of 
different teaching materials and new teaching methods. There can be no educational reform until 
changes occur at the classroom level. Yet, academic teachers indicate greatest need at lower 
levels of concern. Vocational teachers, while aware of the JobReady concept, show intense 
concern at the informational and personal levels of concern. JobReady leaders must market the 
concept to and obtain support from classroom teachers if the initiative is to be sustained and have 
significant impact on students. 

Comparison of educator group SoC profiles reveal that superintendents’, vocational 
directors’, and principals’ lowest concern stage is awareness while consequence and 
collaboration are their highest levels. The implication is that these administrators have become 
knowledgeable about and comfortable with the JobReady reform effort. However, assistant 
principals diverged fi:om fellow administrators and indicated high concern about their role in 
implementing JobReady. JobReady leaders should recognize the importance of assistant 
principals as change agents and identify strategies to educate them on their rote in JobReady 
implementation. 
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Comparison of educator group SoC profiles reveals additional interesting information. 

For example, the intensity level of concerns for superintendents is not statistically similar to 
academic teachers at any stage. Directors are statistically similar to academic teachers at the 
management level only. Higher level administrators consisting of superintendents, vocational 
directors, and principals indicated most intense concern at the consequence and collaboration 
levels. All other groups expressed most intense concern at the personal and informational levels. 
These phenomena should concern JobReady leaders. JobReady cannot reform schools if gaps 
exist between school administrators and those educators responsible for essential career guidance 
and instruction. 

Recommendations 

As local JobReady partnerships become more complex by either increased numbers of 
public school students or additional public school systems, JobReady leaders should allocate 
increased human and material resources. Change strategies must be customized to address 
JobReady needs of all educator groups. To determine what strategies are appropriate, input 
should be obtained from each educator group within each partnership. If only higher level 
administrators are provided mechanisms to advance to higher concern levels while classroom 
teachers flounder at the personal and informational stages, JobReady is likely to become just 
another failed reform effort. 

JobReady leaders should sponsor activities designed to obtain the involvement of 
practitioners in all educator groups involved in JobReady components. Based on the results of 
this study, involvement is an effective means of advancing practitioners to higher concern levels. 
Appropriate information should be disseminated to assistant principals, guidance counselors, 
academic teachers, and vocational teachers. This information should focus on making these 
practitioners more knowledgeable and comfortable with their roles in implementing JobReady. 
Professional development efforts must be ongoing for school-based JobReady practitioners. 
These efforts should be aligned with concerns expressed by the practitioners. For example, the 
particular professional development activity might initially focus on general characteristics, 
effects, and requirements of JobReady. Later efforts might address organizational or 
management needs. 
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Introduction and Theoretical Framework 

Enrollment in agriculture programs at both the university and high school levels has 
undergone major changes over the past twenty years. From the peak numbers of the late 1970s, 
enrollments at the high school level plummeted with the onset of the farm crisis in the late 1970s 
and early 1980s (Dyer & Osborne, 1994). In the two states whose economies are most closely 
tied to the agricultural sector, Illinois and Iowa, some of the most drastic changes were reported. 
In Illinois, agriculture program enrollments decreased by over 60% from a total of 29,502 
students enrolled in 397 programs to 1 1,733 students in 325 programs (Illinois State Board of 
Education, 1993). In Iowa, an enrollment of 17,293 students in 245 programs dwindled to only 
9,161 students in 255 programs from 1976 to 1990, a decrease of nearly 53% (Iowa Department 
of Education, 1997) . Over the same time period, enrollment in Illinois and Iowa public schools 
decreased by 25% and 20%, respectively, indicating that other factors were contributing to the 
attrition in agriculture program enrollments (United States Department of Education [USDE], 
1996). 

This enrollment crisis created a ripple effect that struck at the university level in the late 
1980s. Manderscheid (1988) reported a 24% decline in Land Grant University agriculture 
enrollments and a 13% decrease in non-Land Grant University agriculture enrollments from 
1978 to 1988. Paralleling this decrease in university agriculture program enrollments were 
cutbacks in faculty positions. According to the American Association for Agricultural Education, 
university agricultural education faculty membership decreased from a 1984 high of 326 
members to a 1995 low of 254 members, a decrease of 22% (American Association for 
Agricultural Education, 1996). 

As universities were responding to decreased numbers by downsizing agricultural 
education departments and programs, high school enrollments in agriculture courses were 
rebounding. Several states modernized agriculture curricula as suggested by the National 
Research Council (1988) and reaped almost immediate results in the form of increased student 
numbers. At an enrollment of nearly 12 million students in 1993-94, high school enrollments are 
within 10% of the pre-recession enrollments of 13.6 million students, even with a national 
decrease of just over 20% in the number of school age children (USDE, 1996). 

At the university level, colleges of agriculture are also reporting increased enrollments. 
Litzenberg, Whatley, and Scamardo (1992) reported that with the exception of the North Central 
Region, agricultural enrollments had recovered to early 1980 levels. According to USDE 
numbers, 1992 enrollments in colleges of agriculture nationwide have increased by 18.9% oyer 
the 1981 enrollment of 802,000 students (USDE, 1996). However, the demographic composition 
of today’s agriculture classes has changed from that of the 1980s. Scofield (1995) reported that 
43% of students enrolling in the College of Agriculture at Iowa State University in the fall of 
1994 were from urban backgrounds. According to Russell (1993), this lack of agricultural 
background and/or experience jeopardizes the long term future of the agricultural industry. 
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Russell warned of an impending "brain drain" in the agricultural industry, jeopardizing its long 
term future if the loss to the agricultural industry of individuals trained and experienced in 
agriculture continues. 

With an increasing number of freshmen coming from urban backgrounds and/or 
situations in which they have gained no knowledge of or experience in agriculture, new problems 
and opportunities have emerged. Colleges of Agriculture must provide information, not only m 
agriculture, but also about agriculture. However, losses of enrollments translate into losses of 
dollars from instructional budgets. The needed resources to provide this instruction may not be 
available. 

The problem addressed by this research was how to identify and retain students who are 
likely to complete a program of instruction and seek employment in the industry of agriculture. 
The conceptual model for this study emphasized the need to study those factors that influence a 
student’s selection and pursuit of a field of study and corresponding career choice. Fishbein and 
Ajzen (1975) provided the theoretical framework for this study. They determined that intentions 
to participate in an activity could be predicted based upon knowledge, observation, or other 
information about some issue. This suggested that a person’s intent to pursue study in a field of 
agriculture, or to become actively involved in an agricultural career, may be predicted by 
analyzing his/her beliefs about agriculture. Greenwald (1989) supported this theory, reporting 
that individuals with positive attitudes toward a subject or situation tend to evaluate them 
positively. 



Purpose 

The primary purpose of this study was to assess the attitudes and intentions of Iowa State 
University College of Agriculture freshmen toward high school and university agriculture 
programs and the field of agriculture. A secondary purpose was to investigate student attitudes 
about the role of agricultural educators. The study addressed the following questions: 



1. What were the attitudes of College of Agriculture freshmen toward the field of 
agriculture and educational programs in agriculture? 

2. What were the attitudes of College of Agriculture freshmen toward their major areas of 
study? 

3. What was the influence of high school agriculture program experiences on the attitudes 
of students who are now pursuing agricultural majors? 



Procedures 

This study used a descriptive survey which was administered to all freshmen in the 1996- 
97 class of the College of Agriculture at Iowa State University (N = 513). A student roster from 
the College’s admissions office served as the population frame for this census study. 

Survey instruments were mailed during the 1997 spring semester with a follow-up postcard 
reminder mailed after two weeks. A second instrument packet and reminder postcard were 
mailed at two-week intervals, respectively. A total of 401 (78%) surveys were returned. A 
random sample of ten percent of the non-respondents were contacted and completed the 
questionnaire via telephone. Summated means of these non-respondents were compared to early 
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and late respondents with no significant differences noted, allowing generalization from the 
sample to the population. 

The two-part questionnaire used in this study was developed and used by Dyer, Lacy and 
Osborne (1996) and was reviewed for content and face validity by College of Agriculture staff at 
that time. Part I of the questionnaire addressed demographic information and contained close- 
ended and partially close-ended questions. Part II of the instrument was divided into three 
constructs: Attitudes Toward Agriculture as an Area of Study, Attitudes Toward High School 
Agriculture Programs, and Attitudes Toward University Agriculture Programs. These sections 
used a five-point Likert-type scale (1 = Strongly Disagree, 2 = Disagree, 3 = Uncertain, 4 = 
Agree, 5 = Strongly Agree). Dyer, et al. reported reliability estimates for the three constructs 
using Cronbach’s Alpha (r= .85, .78, .88, respectively). Data were analyzed using descriptive 
statistics, including measures of central tendency and variability. 

Results 

A majority of students respondents was male (57%, n = 222) and Caucasian (97.2%, n = 
369). Afiican- American, Hispanic, and Asian ethnicity/ancestry were reported by five, four, and 
two students respectively (1.3%, 1.0%, and .5%) with an additional seven students (1.8%) 
reporting an ethnicity of "other." 

Fifty-four percent (n = 214) of the respondents reported having completed at least one 
high school agriculture course, 47% (n = 182) indicated that they were FFA members, and 55.2% 
had been involved in 4-H. Nearly two-thirds (62.1%) of those students who had completed high 
school agriculture classes rated the programs as "good" while only 7.5% rated the quality "poor." 
Nearly half of the respondents (n = 188, 47.5%) had farm backgrounds. An additional 28.8% of 
the students were from rural areas (but not farms) or small towns (<1 0,000), with the remaining 
23 .7% of the students indicating their geographical backgrounds were large or medium urban 
areas. 

The majority of the respondents (66.4%, n = 263) indicated they had both paid and 
unpaid experiences in agriculture. Forty-six respondents (1 1.6%) indicated unpaid work 
experience only, 22 students (5.6%) reported paid work experience only, while 65 students 
(16.4%) indicated they had no agricultural experience prior to enrolling in the College. 

Sources of information cited as most helpful in informing students about the College of 
Agriculture and/or Iowa State University included brochures (32.2%), campus visits (24.0%), 
letters from staff (20.7%), phone calls (9.4%), and personal contacts from faculty (6.7%). 

Research Question 1 : What were the attitudes of College of Agriculture freshmen toward the 
field of agriculture and educational programs in agriculture? 

The attitudes of College of Agriculture freshmen toward the field of agriculture were 
generally positive. As indicated in Table 1, students viewed the field of agriculture as both 
scientific and technical. They disagreed with the statement that only students with farm 
backgrounds should pursue agricultural careers (M = 1.55). While less than half (45.9%) agreed 
that most people have a positive image of agriculture, 70.7% agreed that the image was 
improving. 
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Table 1 

Attitudes Toward Agriculture as an Area of Study 



Statement 


Agree* 

f(%) 


Uncertain i 
f(%) 


Disagree* : 

f(%) : 


Agriculture is a scientific area of study. 


371 (93.7) 


19(4.8) 1 


6(1.6) i 


Most people have a positive image of i 

agriculture. 


182(45.9) : 


108 (27.3) 1 


106(26.7) 1 


Agriculture is a highly technical field of 
study. 


343 (96.7) 1 


44(11.1) I 


9 (13) 


The image of agriculture is improving 


280(70.7) i 


88 (2i2) i 




Only students with farm backgrounds should 
pursue careers in agriculture. 


13 (3.3) 


22 (5.6) j 


361(91.1) : 



*The term "agree" includes the combined responses of "strongly agree" and "agree". The term 
"disagree" includes the combined responses of "strongly disagree" and "disagree." 



Respondents were very supportive of high school agriculture programs (Table 2). Over 
three-fourths (75.7%) of freshmen in the College of Agriculture agreed that more students should 
be encouraged to take high school agriculture classes. Accordingly, over two-thirds agreed 
(68.5%) that high school agriculture courses were applicable to college preparatory programs 
and that these courses were good preparation (58.7%) for studying agriculture in college. The 
students disagreed with the statements that that high school agriculture courses are better suited 
for male students and that only students pursuing careers in agriculture should enroll in high 
school agriculture. 
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Table 2 

Attitudes Toward High School Agriculture Programs 



Statement 


Agree* 

f(%) 


Uncertain 

f(%) 


Disagree* 

f(%) ! 


More students should be encouraged to enroll 
in high school agriculture programs. | 


299 

(75.7) 


76 (19.3) 


20(5.1) i 


College-bound students should be encouraged j 
to enroll in high school agriculture programs. 


271 

(68.5) i 


95 (24.0) 


30(7.6) i 


High school agriculture is good preparation for ■ 
college study in agriculture. i 


231 

(58.7) 


137(34.8) 


26(6.6) i 


High school agriculture should become less 
vocational. 


80 (20.3) 


207(52.5) i 


107 

(27.2) 1 


High school agriculture should become more 
scientific. 


172 

(43.6) ; 


176(44.7) ! 


46(11.7) : 


High school agriculture classes are better suited i 
to male students. | 


65(16.5) I 


104(26.3) 1 


225 

(57.1) : 


High school study in agriculture is easier than 
in other subjects. 


131 

(33.4) : 


177(45.0) 1 


85(21.6) I 


High school agriculture courses are beneficial 
for higher-achieving students. 


151 

(38.5) 1 


197(50.3) 1 


44(11.2) : 


High school agriculture courses are beneficial 
for lower-achieving students. 


138 

(35.0) : 


201(51.0) 1 


55(13.9) : 


Most high school students should take some 
course work in agriculture. 


240 

(60.6) 1 


102(25.8) I 


54(13.7) : 


Only students pursuing careers in agriculture 
should enroll in high school agriculture 
courses. 


39(9.9) ( 


59 (14.9) 1 


298 

(75.3) : 



*The term "agree" includes the combined responses of "strongly agree" and "agree". The term 
"disagree" includes the combined responses of "strongly disagree" and "disagree." 



Students generally believed that everyone would benefit from college course work in 
agriculture (see Table 3). Approximately two-thirds (66.1%) of the respondents agreed with the 
statement that more students should be encouraged to enroll in college agriculture courses. 
Likewise, 66.7% of the freshmen disagreed with the statement that only students pursuing 
careers in agriculture should enroll in college agriculture courses. More than three-fourths 
(76.1%) of the students surveyed disagreed that college agriculture courses are better suited to 
male students. 
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Table 3 

AttitudesToward University Agriculture Programs 



Statement 


Agree* 

f(%) ) 


Uncertain i 
f(%) ; 


Disagree* 
f(%) : 


More students should be encouraged to enroll 
in university agriculture programs. 


261 (66.1) j 


112(28.4) 


22 (5.6) i 


College agriculture classes are better suited, to 
male students. 


19(4.9) i 


75(19.0) 1 


300(76.1) ! 


College study in agriculture is easier than in 
most other subjects. 


42(11.9) ; 


93 (23.8) : 


224(64.3) ; 


Most college students should take some course 
work in agriculture. 


191 (48.3) i 


128(3^3) 


77(19.5) 1 


Only students pursuing careers in agriculture 
should enroll in college agriculture courses. 


66(16.6) 1 


66(16.7) i 


264(66.7) : 



*The term "agree" includes the combined responses of "strongly agree" and "agree". The term 
"disagree" includes the combined responses of "strongly disagree" and "disagree." 



Research Question 2: What were the attitudes of freshmen College of Agriculture students 
toward their major area of study? 

Nearly all students indicated they were happy in the College of Agriculture. Only 5.9% of 
the respondents indicated they were planning a change of colleges (see Table 4) and only 6.5% 
intended to enroll in another university. Likewise, most respondents (n = 294, 82.7%) reported 
no intent to change majors. Thirty-one students (8.8%) indicated a definite plan to change of 
majors, whereas 30 students (8.5%) indicated a possible change. 

Table 4 

College of Agriculture Freshmen Planning to Change Colleges and/or Majors 







Students i 

contemplating ! 
change of college i 


Students 
contemplating 
change of majors 


Major 


N : 


n 


% ; 


n 


% 


Agricultural Systems Technology i 


16 1 


0 


0.0 i 


.2 ! 


12.5 : 


Agronomy 


16 ] 


0 


0.0 ; 


3 ; 


18.7 


Animal Science, Pre-Vet, Dairy Science j 


135 1 


5 


3.7 j 


20 i 


14.8 


Horticulture, Turfgrass Management 


25 1 


0 


0.0 ; 


4 : 


16.0 ; 


Agricultural Studies 


32 


0 


0.0 i 


3 i 


9 4 ^ 


Agricultural Business 


51 : 


5 


9.8 1 


6 : 


11.8 
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Table 4 (continued) 

College of Agriculture Freshmen Planning to Change Colleges and/or Majors 







Students 

contemplating | 
change of college ! 


Students 
contemplating 
change of maiors 


Major 


N i 


Q 


% ! 


D 


% 


Genetics, Zoology 


11 ; 


2 


18.8 i 


4 1 


36.4 


Agricultural Education 


15 : 


0 


0.0 ; 


1 I 


6.7 


Agriculture-Undeclared 


14 1 


4 


28.6 ; 


7 1 


50.0 


Forestry 


12 I 


1 


8.3 1 


4 i 


33.3 


Animal Ecology 


18 1 


3 


16.7 i 


1 i 


5.6 i 


Microbiology 


13 ! 


2 


15.4 ; 


3 i 


23.1 ; 


Food, Nutritional Science 


4 ; 


0 


0.0 i 


1 i 


25.0 1 


Public Service Administration 


. 3 


0 


0.0 : 


1 i 


33.3 : 


Environmental Science 


i 


0 


0.0 ; 


0 1 


ao 


Biochemistry 


2 : 


0 


0.0 j 


1 1 


50.0 : 


Totals 


350 


22 


5.9 1 


61 I 


3 ^ 



Research Question 3 : What was the influence of high school agriculture program experiences on 
the attitudes of students who are now pursuing agricultural majors? 

A majority (n = 214, 54%) of the College freshmen reported course work in high school 
agriculture whereas 1 82 students (46%) reported no high school course work in agricultural 
education. Of those who did not complete agriculture course work, 68.7% did not have access to 
a high school program. The second most cited reason was a conflict with college preparatory 
courses. 

Almost all students (n = 197, 97.0%) from high school agriculture programs indicated 
that they intended to graduate from the College of Agriculture and pursue a career in agriculture. 
Only six students indicated otherwise. By comparison, 90.5% (n = 152) of those students who 
did not enroll in high school agriculture programs indicated that they plan to graduate in the 
College of Agriculture. Sixteen students indicated intents to change colleges. 

Of the 169 students who had been enrolled in high school agriculture programs and were 
also FFA members, 166 (98.2%) indicated intentions to graduate with a degree in the College of 
Agriculture. Likewise, of the 198 students who had been 4-H members, 192 (97.0%) also 
indicated intentions to graduate through the College of Agriculture. 

Students who had completed high school agriculture courses expressed more positive 
attitudes toward educational programs in agriculture (see Table 5). In addition, these 
students disagreed (M = 1 .62) with the statement that only students with farm 
backgrounds should pursue careers in agriculture. Students with no high school agriculture 
course work strongly disagreed with the statement (M = 1 .43). 
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Table 5 

Comparison of Attitudes of High School Agriculture Program Versus Non-Program Graduates 





No High School 
Agriculture 




High School 
Agriculture 


Statement 


M 


SD 




M 


SD 


Agriculture is a scientific area of study. 


4.37 ; 


.72 




4.37 : 


.61 


Most people have a positive image of 
agriculture 


3.19 


1.00 




3.28 j 


.99 


Agriculture is a highly technical field of 
study. 


4.07 


.72 




4.19 ; 


.67 


The image of agriculture is improving. 


3.77 


.82 




3.87 


.84 


More students should be encouraged to enroll 
in university agriculture programs 


3.63 I 


.80 




3.92 


.81 


More students should be encouraged to enroll 
in high school agriculture programs 


3.73 


.91 




4.37 ; 


.77 


College-bound students should be 
encouraged to enroll in high school 
agriculture courses. 


3.68 


.97 




4.11 


.89 


High school agriculture is good preparation 
for college study in agriculture. 


3.46 


.85 




4 J3a 




High school agriculture should become less 
vocational. 


3.05 


.69 




2.82 i 


.96 


High school agriculture should become more 
scientific. 


3.39 


.73 




3.37 


.91 


Only students with farm backgrounds should 
pursue careers in agriculture. 


1.43 


.64 




1.63“: 


.85 


High school agriculture courses are better 
suited to male students. 


2.46 


1.08 




2.20 : 


1.14 


College agriculture courses are better suited 
to male students. 


1.96 


91 




1.87 : 


.81 


High school study in agriculture is easier than 
in most other subjects. 


3.00 


.63 




3.29 


1.12 


College study in agriculture is easier than in 
most other subjects. 


2.29 


.94 




2.27 : 


.96 


High school agriculture courses are beneficial 
for higher-achieving students. 


3.10 


.60 




3.51^ 


.91 


High school agriculture courses are beneficial 
for lower-achieving students. 


3.01 


.68 




3.44 


.93 


Most high school students should take some 
course work in agriculture. 


3.35 


.87 




3.82“ 


.90 



98 



109 



Table 5 (continued) 

Comparison of Attitudes of High School Agriculture Program Versus Non-Program Graduates 



49), 





No High School , 
Agriculture 




High School 
Agriculture 


Statement 


M 


SD \ 




M 


SD 


Most college students should take some 
course work in agriculture. 


3.26 


.93 




3.43 : 


.99 j 


Only students pursuing careers in agriculture 
should enroll in high school agriculture. 


234 


.84 




ioo 


.89 


Only students pursuing careers in agriculture 
should enroll in college agriculture. 


2.45 


.91 i 




2.30 


.91 ; 



- — - ; - - ' 

Means were categorically different. Categories of agreement were: "Strongly Disagree" 



1.00-1.49), "Disagree" (M = 1.50-2.49), "Uncertain" (M = 2.50-3.49), "Agree" (M = 3.50-4 
and "Strongly Agree" (M > 4.50). 



Students indicated that the most influential person in their decision to attend the College 
of Agriculture was their high school agriculture teacher (20.0%), followed by high school 
guidance counselor (16.4%), sibling (10.9%), parent (7.3%), and friend (5.5%). For those 
students who had decided to transfer out of the college/university, most common influences were 
relatives/friends (25%), teachers (10%), perceived lack of job opportunities (10%), academic 
pressure (10%), and not feeling part of the program (10%). 

Conclusions, Recommendations, and Implications 

Nearly all students indicated they were happy in the College of Agriculture. Less than 6% 
of the respondents indicated a likely change of colleges. These numbers contrasted sharply with 
those reported by Dyer, Lacey, and Osborne (1996). Dyer, et al. reported that nearly 40% of the 
freshmen class in the College of Agriculture at the University of Illinois at Urbana-Champaign 
intended to leave the College of Agriculture prior to graduation. In addition, the researchers 
reported that nearly 50% of the U of I freshmen intended to change majors within the College as 
compared to 17.3% at Iowa State. Why do students choose to remain in one college but leave 
another? Given the geographical and peer comparisons between the two institutions, further 
study should contribute to understanding students’ rationale. 

The College of Agriculture at Iowa State University continues to attract students with 
farm or rural backgrounds and who have completed high school agriculture courses. The 
researchers in this study found that the percentage of students coming from large or medium 
urban areas had declined from 43% in 1995 to 23.7% in 1996. This finding supports a recent 
article appearing in the Des Moines Register (Pins, 1 997) which indicated that a "rural rebound" 
is occurring throughout the United States as three-fourths of the nation’s 2,304 non-metro 
counties are growing. This is in contrast to growth rates in the 1980s when less than half of 
Iowa’s counties experienced an increase in population. If the same phenomenon is occurring 
nationwide, this increase in population in rural areas should fuel enrollment and resources into 
colleges of agriculture and provide even greater demand for additional high school agriculture 
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programs. Major efforts and support from universities should be expended in providing 
assistance in establishment of those programs. 

High school agriculture programs are good investments by those interested in promoting 
agriculture. Likewise, graduates of these programs are good investments by colleges of 
agriculture. Students who have enrolled in high school agriculture, and those who were involved 
in FFA and/or 4-H, are more likely to complete a four-year degree in the College of Agriculture 
and choose agriculture as a career than are freshmen who have not had those experiences. The 
research indicated that 97% of students that enrolled in high school agriculture plan to graduate 
from the College of Agriculture and pursue a career in the industry. This percentage increased 
slightly if students were also involved in FFA and remained constant for those involved in 4-H. 
The College of Agriculture should target high school agriculture programs in its recruitment 
program. By directing recruitment efforts toward this group of students, the College of 
Agriculture can make more efficient use of resources and increase retention rates. 

Many secondary students in Iowa do not have the opportimity to enroll in high school 
agriculture programs. This study found that of those students not enrolling in high school 
agriculture, 68% did not have access to such a program. The second most cited reason was a 
conflict with college preparatory courses. The number of high school agriculture programs in the 
state should increase so that all students have access to agricultural experience, and classes 
should be offered at times that will minimize scheduling conflicts. An increase in the number of, 
and enrollment in, high school agriculture programs, should provide the College of Agriculture 
even greater enrollment and retention opportunities. 

High school agriculture teachers continue to play an important role in assisting students 
in determining whether or not to pursue a college degree. Twenty percent of the respondents 
indicated their agriculture teacher was the most influential source in making this decision. High 
school guidance counselors, siblings, and parents were respectively mentioned as the most 
influential person by 16.4%, 10.9%, and 7.3% of the respondents. The College of Agriculture 
should recognize the importance of teachers of agriculture and continue to nurture these 
relationships. In addition, the College should keep these individuals informed of program 
changes and related information. By maintaining a positive relationship with teachers, 
counselors, parents, and siblings, the College can be more effective in its recruitment efforts. 

The College of Agriculture should devote adequate resources to brochure development 
and distribution. Over 32% of the freshmen indicated that brochures were the most helpful 
source of information. Campus visits and letters from staff were respectively ranked by 24% and 
20.7% of the respondents as the most effective source of information. Additionally, 6.7% of the 
respondents indicated that personal contacts from faculty were useful. The College should 
continue to provide quality recruitment brochures and encourage campus visits as part of its 
recruitment plan. 

The College of Agriculture should explore the use of alternative retention activities (such 
as learning communities/teams, etc.) for incoming freshmen. For those students who had decided 
to transfer out of the college/university, 10% indicated their reason as not feeling part of the 
program. While learning teams have only been in existence in the College of Agriculture for two 
years, anecdotal evidence indicates that these teams help ease the transition to college life by 
creating a "family" away from home. Further research to determine the usefulness of these teams 
is warranted. Likewise, a follow-up study should be conducted during respondents’ senior years 
to determine if respondents have followed through on their stated intentions. 
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Introduction and Theoretical Framework 

The most important consideration of any laboratory teacher should be the provision for 
safety of all participants. What and how students learn must be secondary to the physical safety 
of both students and teachers in agricultural education laboratories (Bruening, Hoover, & 
Radhakrishna, 1991). 

Parents place a premium on the safety of their children. They expect their children to be 
educated in the proper and safe use of all tools, equipment, and materials which the student will 
use. Even in instances where all precautions have been taken, the potential for serious injury still 
exists. Not only is safety an important consideration for educators, but a moral, professional, 
and legal obligation as well (Daniels, 1 980; Gliem & Hard, 1 988). The primary responsibility 
for providing safety instruction and a safe learning environment rests with the teacher 
(McMahon, 1975). 

While programmatic research is expected across disciplines, it is imperative that study in 
this area of inquiry focus on the problems associated with agricultural education laboratory 
safety and address those problems in an orderly, systematic approach. In doing so, the need 
existed for a thorough review of research to critically examine its status and provide the 
profession with a basis from which to direct future research efforts. 

The theoretical framework for this study was furnished by Williams (1991) in his 
“Dimensions of Agricultural Education” model. Williams noted that “we must fully understand 
the dimensions of agricultural education before we can successfully focus our research” (p. 8). 
Review and acknowledgment of past research, coupled with identification of voids in the current 
knowledge base is an effective method of fostering that understanding and bringing focus. 

Purpose and Objectives 

The purpose of this investigation was to synthesize research related to safety in high school 
agricultural education laboratories. The study was guided by the following objectives; 

1. Identification of research in agricultural education laboratory safety. 

2. Identification of deficiencies in research related to agricultural education laboratory 
safety. 
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Procedures 



Historical research methods were utilized to accomplish the objectives of this study. Four 
sources were used to gather data to meet the objectives of the study: Journal of Agricultural 
Education. Dissertation Abstracts International. Proceedings from Regional and National 
Agricultural Education Research Meetings, and ERIC Documentation Reproduction Service. 
Studies appearing in these references were located through a library systems search completed at 
two Midwestern universities and consisted of articles published through May, 1997. 

Findings 

Objective 1: Identification of research in agricultural education laboratory safety. 

Occurrence of Accidents 

Agricultural education laboratories are potentially hazardous learning environments. In 
studies involving high school agricultural education programs. Swan (1993) reported a mean of 1.3 
major (requiring medical attention) student accidents and 13.3 minor accidents (requiring bandage 
but not medical attention) per year. Lawver (1992) reported a mean of 0.8 major student accidents 
per year, with a range of 0-13 accidents per teacher. Nearly 42% of the respondents reported at 
least one accident. A second study by Lawver and Fraze (1995) reported a mean of 3.19 injuries 
which required medical attention. 

Gliem and Miller (1993b) conducted a study involving schools which had agriculture 
laboratories and had been engaged in litigation resulting from student injury. The researchers 
reported the range in number of injuries as 0-14. Hard (1990) reported a majority of accidents 
occurring primarily in the woodworking or welding/metals area. 

Teacher Liability 

While many agricultural educators may be doing a good job of instructing students in safety, 
many others are vulnerable to legal action if an injury occurs. According to Reece (1980) and 
Laird and Kahler (1995), agriculture instructors agree that laboratory safety is important. 

Likewise, Gliem and Miller (1992a) reported that 100% of the administrators responding to their 
survey indicated that teachers instruct students in how to properly use equipment and 
demonstrate that usage, and 97.7% give students an equipment test. However, while instructors 
perceive written documentation to be important (Johnson & Schumacher, 1988, 1989), they 
generally fail to maintain written reports (Lawver, 1992; Swan, 1993). Of concern, Gliem and 
Miller reported that 15.6% of the administrators in their study reported that teachers leave the 
laboratory while students are using dangerous equipment or chemicals. 

Teachers may be lax in their responsibility to know and conform to local, state, and national 
safety laws. Bruening, Hoover, and Radhakrishna (1991) reported the greatest deficiency of 
Pennsylvania agricultural mechanics teachers was knowledge of state safety laws. Reneau and 
Poor (1983) reported that teachers only scored 68.2% on a liability survey given to determine 
teachersD knowledge of liability issues. Rudolph and Dillion (1984) noted that only 17 of the 37 
safety standards were being met by more than 50% of the agricultural education departments in 
their study. Hard (1990) reported that nearly 50% of the teachers in his study did not know the 
recommended guidelines for noise, lighting, or ventilation requirements of their laboratories, 

51% considered their tools unsafe to operate at times, and nearly 90% believed themselves to be 
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careless in the laboratory. According to Hard and Miller (1990), whereas nearly all teachers 
(97%) considered knowledge of safety laws important, 24% indicated they did not know the 
laws. 

Fletcher and Johnson (1990) concluded that teachers were not using recommended safety 
practices or providing student safety and emergency equipment to the extent warranted by the 
hazards present in the agricultural mechanics laboratory. These conclusions supported earlier 
findings by Lamb (1985) in a Missouri study. 

It may be possible to identify students and/or teachers who are potential safety risks prior to 
their entering a laboratory setting. Indicators such as low emotional and behavioral control 
(Pulkkinen, 1995) and the identification of students with low self-esteem (Bettis & Crawford, 
1972) may be used to identify students likely to be safety risks. The researchers recommended 
that students who were identified as accident repeaters be given additional attention or safety 
instruction. Plummert (1995) found that students overestimate their physical abilities and noted 
that this could be related to their accident proneness. 

Availability of Safety Equipment 

Bennett (1984) concluded that providing a safe learning environment for public school 
agricultural education programs is a problem in most schools. Swan (1992) reported that 
instructors were not using recommended safety practices, and that school systems were not 
providing students with adequate safety equipment. Gliem and Miller (1993 a) also reported the 
availability of safety equipment/materials to be lacking in some schools. 

Concerning availability of safety equipment, researchers have provided similar lists of 
safety equipment found in agricultural mechanics laboratories. The most frequent available 
equipment/materials are fire extinguishers, welding gloves, screens, booths, exhaust systems, fire 
alarms, safety glasses, safety guards, first aid kits, exit signs, posted eye safety regulations, safety 
posters at power tools, safety zones, and safety cans for flammable liquids (Bruening, Hoover, & 
Radhakrishna, 1991; Fletcher & Johnson, 1990; Lawyer, 1992; Swan, 1993). Panic buttons were 
the least frequently reported safety item by Swan (1993). Bennett (1984) listed the most 
common safety hazards found in Mississippi high schools to be: lack of exit signs, equipment 
without guards, ungrounded extension cords, absent or uninspected fire extinguishers, and 
inadequate paint storage. 



Instructional Techniques 

Instructors use a variety of materials while teaching safety, but limit their instructional 
repertoire to demonstration. Instructors used safety exams, manuals, safety related subject 
matter, clean-up schedules, inspections, booklets, videotapes, and worksheets most often as 
instructional materials/practices (Bruening, Hoover, & Radhakrishna, 1991; Lawyer, 1992; 

Swan, 1993). Teachers listed the operation of power tools, eye protection, use of hand tools, and 
electrical safety as the most important safety instructional topics (Lawyer). 

Teacher Preparation 

The role of teacher educators is important in developing the safety proficiency of pre-service 
and current teachers (Gliem & Miller, 1993a). However, studies have indicated several voids 
exist in teacher preparation in laboratory safety (Forsythe, 1982, 1983; Forsythe & Terry, 1981; 
Jarrett, 1967; Rosencrans, 1996). In national studies of teacher educators, Forsythe (1982, 1983) 
and Forsythe and Terry (1981) concluded that teacher educators provide minimal experiences and 
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instruction designed to develop teacher competency in safety. 

Another void exists in the amount of continuing instruction provided to experienced 
teachers. According to Bennett (1984) and Lamb (1985), teachers with less experience place 
more emphasis on laboratory safety than do experienced teachers. Likewise, Schumacher and 
Johnson (1990a, 1990b) reported the average agriculture teacher in Missouri placed less 
perceptual importance on safety-related competencies than did student teachers. Additional 
training in the form of inservice workshops, training seminars, or additional course work has 
been recommended by Everett (1982); Fletcher and Johnson (1990); Forsythe and Terry (1981); 
Hard and Miller (1990); Johnson, Schumacher, and Stewart (1990a, 1990b); and Umbaugh 
(1989). 



Administrators’ View of Safety 

Safety is important to administrators (Gliem & Miller, 1992a, 1992b, 1993b). They 
overwhelmingly agree that safety instruction should be included as part of the curriculum 
(1992b, 1993b). However, most administrators depend on the knowledge and expertise of 
teachers in providing a safe environment and believe that teachers are responsible for the repair 
and maintenance of laboratory equipment (1992b, 1 993b). 

There is an apparent connection to administrators’ attitudes toward laboratory safety and 
teacher actions. Gliem and Miller (1992a) reported significant positive correlations between 
administrator’s safety attitude and the teacher’s preparedness to provide safety instruction, the 
number of safety materials/equipment available, and the number of teacher safety practices used. 

Student Attitudes Toward Safety 

Students’ safety attitudes are significantly and positively related to their perceptions of 
instructor and parental safety attitude (Hard, 1990; Harper, 1984; Harper & McCracken, 1984); 
their agricultural mechanics safety knowledge; employer’s safety attitudes; teacher’s, parent’s, 
and employer’s safety knowledge; and school, home, and workplace safety environment (Harper, 
1984; Harper & McCracken, 1984). 

Instructor Attitudes Toward Safety 

Laird and Kahler (1995) reported that instructors considered safety to be the most important 
unit of instruction in agricultural mechanics laboratories. Maines (1989) reported the antecedent 
variables explaining instructor safety attitudes were instructor age, years of teaching experience, 
the instructor’s having had a serious accident, and/or having had a close friend or relative killed 
in a work-related accident. Furthermore, Maines identified the best predictors of instructor 
safety attitude as the instructor’s perception of his/her health hazard knowledge base, the 
instructor having had a serious accident in the past, and the instructor having had horticulture as 
a specialty area. Lawyer and Fraze (1995) reported inconsistencies between teachers’ attitudes 
toward safety and the teaching of those practices. 

Noise levels 

Some studies have reported that while noise is a concern, it is not a problem (Madou- 
Bangurah, 1978; Shell, 1972). Others have examined effects of noise on students and warn of 
the dangers of noise created in these learning environments (Miller, 1986, 1988, 1989; Miller, 
Jacobs, & Schimpp, 1992; Reynolds, 1990; Weston & Stewart, 1980; Woodford, Lawrence, & 
Bartrug, 1993). 
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Miller (1988, 1989), Reynolds (1990), and Miller, Jacobs, and Schimpp (1992) all 
concluded that both students and teachers are routinely experiencing significant and potentially 
damaging noise exposure during welding in school laboratories. Jewell (1979) noted that as 
noise intensities increased in the learning environment, the amount of time necessary to complete 
a task also increased. Miller (1986) concluded that performance loss in cognitive and motor 
domains may be reduced by providing effective hearing protection devices in noisy 
environments. However, Miller and Schimpp (1993) noted that the intensity of noise is not the 
primary factor in reduction in performance, but rather the interaction from the nature of the 
noise. They recommended teachers determine the nature of noise, rather then try to eliminate all 
of it. 

Teachers are also susceptible to hearing problems. Burke (1987) reported hearing loss 
greater in agricultural education teachers than that observed in other people of the same age 
group. Woodford, Lawrence, and Bartrug (1993) noted that all of the instructors in their study 
had high frequency loss of hearing greater than 25dBHL. 

Ventilation 

Only one study was found which mentioned the safety problems associated with 
ventilation/dust/smoke. Madou-Bangurah (1978) noted that in laboratories where smoke and/or 
dust were problems, either no ventilation or improperly installed ventilation was the cause. 



Objective 2: Identification of deficiencies in research related to agricultural education 
laboratory safety. 

In analyzing the findings from this research, several areas of deficiency were found. Future 
research should seek to answer the following questions: 

1. What factors influence teachers’ attitudes toward safety? 

2. How do teachers acquire safety knowledge and develop positive safety attitudes? 

3. What factors contribute to the variance in laboratory? 

4. How should safety training differ for the varying types of agricultural laboratories? 

5. Which teaching methods are most effective in generating student motivation towards safety? 

6. Which laboratory activities constitute the greatest hazards to students and teachers? 

7. What demographic factors influence student and teacher safety knowledge, attitudes, and 
practices? (i.e., geographic background, ethnicity, age, grade level, gender) 

8. Is there a relationship between level of achievement and safety attitude? 

9. How do teacher record keeping practices affect the propensity for litigation? 




106 117 



10. What are the inservice needs of teachers? 

11. What are the major causes of serious and/or minor accidents in the laboratory? 

12. Are there curricula which are not appropriate for high school agricultural laboratory 
instruction because of the danger involved? 

13. What are the characteristics of individuals most likely to be safety risks in the laboratory 
setting? 

14. What are the dangers of prolonged exposure in a laboratory environment to agriculture 
teachers? 

15. What is the effect of cumulative noise (home, school, work) on studentsD hearing and/or 
performance? 

Conclusions and Recommendations 

Overall, research addressing the problem of safety in agricultural mechanics laboratories can 
best be characterized as a mixture of descriptive, survey, and experimental studies. To a greater 
extent than most other areas of agricultural education, research in the area of agricultural 
mechanics has also been relatively programmatic. In areas examining instructional methods and 
the effects of noise, a sound empirical base of knowledge has been established from which future 
research efforts may be directed. 

Agricultural education laboratories are potentially hazardous places for both work and 
study. Due to the nature of these laboratories, the inexperience of students who participate, and 
the proximity to dangerous equipment and chemicals, the potential for injury exists. Teachers 
should make every effort to minimize these risks by preparing themselves to effectively and 
safely manage these laboratories and to provide first aid assistance if needed. Likewise, 
administrators should assume a more active role in monitoring laboratory safety and in the 
procurement of necessary safety equipment and/or materials. 

While teachers claim to support the concept of laboratory safety, many fail to fully 
implement safety guidelines and practices to the extent warranted by the hazards present. This 
failure results in increased danger to both students and teachers, and increased liability to 
teachers. Strategies to involve all partners (students, teachers, parents, and administration) in 
laboratory safety programs should de developed and tested. 

Generally, teachers appear to be remiss both in their knowledge of local, state, and national 
safety laws and in their performance in providing a safe environment for themselves and their 
students. In nearly all studies reviewed, safety violations were noted. Likewise, a serious void 
exists in teacher safety preparation. Pre-service teachers are leaving colleges without being 
adequately trained in first aid measures or prepared to safely manage agricultural laboratories. 
Experienced teachers are even less safety conscious. Furthermore, no research showed that local 
school systems are addressing the safety-related deficiencies of their employees. A safety 
education course, complete with first aid certification, should be a requirement by all teacher 
education programs. Pre-service and inservice workshops/seminars should be incorporated into 
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teacher education programs to more adequately prepare teachers to address safety-related 
concerns. 

The possibility may exist to identify students and teachers who are possible safety risks 
prior to their entering a laboratory setting. Research in this area should be further explored in 
pursuit of the development of an instrument that could be used throughout the teaching 
profession. 

Noise levels typically found in agricultural mechanics laboratories constitute a nuisance, 
affect performance, and may be dangerous to students and teachers. Because of student and 
teacher exposure of high levels of noise outside the laboratory setting, every effort should be 
made to limit exposure during laboratory classes. All students and teachers should wear hearing 
protection devices during laboratory classes in which noise levels approach OSHA limits. 

Most of the research conducted to date focuses on the hazards of the laboratory environment 
to students. Additional research is needed concerning the prolonged exposure to teachers in 
laboratory settings. 
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INTRODUCTION 

The phenomenon of increasing numbers of women choosing nontraditional occupations 
has evolved most noticeably during the last two decades. Social, political and economic factors 
have combined to give females more choice in selecting their type of work. 

Lynn Shaw, an electrician profiled in Blue Collar Women: Trailblazing Women Take on 
Men-Onlv Jobs (Ferguson, 1994), expressed her feelings about nontraditional work in the 
following way. 

I think the biggest significance of women working in nontraditional work is breaking 

down gender roles. People need to get jobs on the basis of interest and abilities. In 

our society, there are strict race and gender lines. Only a few people are doing this work. 

Gender roles are very limiting. Increased numbers will break down the perception that 

this is man’s work.(p. 115) 

In the United States, more women are in the workforce than ever. The 1995 Statistical 
Abstract records the participation rate of females in the civilian workforce as 58.8%, up from 
57.5% in 1990, and 51.5% in 1980 (U.S. Bureau of the Census). In 1900, Just one in five women 
was working for wages (Macionis, 1994). Although the range of occupations is expanding as 
more women work, the type of work women choose is still affected by strong cultural and 
historical traditions and constraints. 

Women earning baccalaureate and advanced degrees have noticeably Joined the white 
collar ranks. For example, between 1972 and 1990, the proportion of female lawyers to total 
lawyers rose from 4% to 21% and female physicians rose to 38%, nearly double their 1972 
proportion (U.S. Bureau of Labor Statistics, 1991). 

However, significant numbers of women still work in clerical and service Jobs which are 
predominately female, and offer wages considerably less than the male Jobs requiring the same 
level of education. The skilled trades and technical occupations are overwhelmingly male. For 
example, 98.1% of masons and metal workers are male, as well as 70.1% of computer specialists 
(U.S. Bureau of Labor Statistics, 1994). Jobs in these trade and technical areas offer wages 20- 
30% higher than comparable female-dominated occupations like health-care (Bleakley, 1994). 

The term nontraditional in this study refers to occupations in which women comprise 
25% or less of the population (Schwartz and Zimmerman, 1992). Both vocational-technical and 
professional occupations are classified in this manner. 

Significance of the Study 

As more women enter the workforce as primary, or much needed secondary, household 
income providers, the factors affecting their occupational choice are of interest to educators. 




114 



125 



social scientists, politicians and the business community. Recently, nontraditional jobs for 
women have become very related to the controversies surrounding single mothers and welfare. 
Most of the jobs for which poor women are qualified pay poorly. A move away from low paying 
traditionally female work is crucial as the demographics of poverty are examined. 

In response to the many welfare programs concerned with only getting jobs of any type, 
economist Rebecca Blank stated “I am extremely concerned about asking women to make it in a 
labor market where they can’t earn enough to support their family” (Trammel, 1995, p.30). 

Malloy (1993) also voiced concern that women “are still training for jobs which will not 
enable them to achieve their goals of a financially rewarding career” (p.l02). Therefore, even in 
the face of long-standing genderized occupations, it seems inevitable that interest in 
nontraditional training will continue in the future. 

Statement of the Problem 

What factors influence females choosing nontraditional vocational-technical occupations? 

Other related and specific research questions included: 

What demographic differences exist between females choosing traditional vocational- 
technical occupations and those choosing nontraditional vocational-technical occupations? 

What factors are considered advantages and disadvantages of traditional and 
nontraditional occupational choices by females choosing those occupations? 

What are the perceived barriers for females entering nontraditional occupations according 
to those choosing such careers as well as staff members at training institutions with 
nontraditional students? 

What opinions do staff and nontraditional students hold regarding recommending 
nontraditional occupations for females? 

REVIEW OF RELATED LITERATURE 
Social and Economic Issues 

The changing demographic and social character of the U.S. workforce has been 
influenced by a number of factors including, the decline of family farms, the growth of cities, the 
rising divorce rate, increasing numbers of single mothers, decreasing family size, and changing 
societal norms regarding appropriate roles for women. 

Recent economic stagnation and past inflationary periods have reduced the real income of 
many families in the U.S. These conditions have in turn increased the number of two-income 
families to almost 60% (U.S. Bureau of Labor Statistics, 1994). Fifty-nine percent of all married 
women work outside of the home, as do 60% of married women with children under six. Sixty 
percent of divorced women with children under six work, and 80% of divorced mothers with 
older children work (U.S. Bureau of Labor Statistics, 1994) 

Since the 1970s, the number of single working mothers has dramatically increased. In 
1990, 5.8 million single mothers were working compared with 4.4 million working in 1981, and 
2.1 million in 1971. The economic fate of many families headed by women is bleak. The 
feminization of poverty as been identified as a troubling American trend. “Of all poor families, 
52.4 percent are headed by women with no husband present, while just 5.9 percent of poor 
families are headed by single men” (Macionis, 1994, p.284). 
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An important issue is that of childcare. Single mothers in all occupations find it difficult 
to provide affordable and safe daycare, but nontraditional training and work schedules often 
present unique challenges. A JTPA service provider reported that the nontraditional fields are 
“set up for people who have a wife at home” (Pearce, 1993, p.34). Another service provided 
stated that unless a prospective student has relative that is going to be available on a 24-hour 
basis, that student won’t be considered for nontraditional training. Pearce (1993) stated that 
many stories of failures of women entering nontraditional training revolved abound “the collapse 
of a child care arrangement” (p.34). 



Legislative History 

Legislative reforms have evolved in an effort to overcome the sex segregation that 
historically has typified the American occupational structure. In 1984, significant legislation 
affecting vocational education overall was passed. The Carl Perkins Act expanded vocational 
education by increasing the number of special populations to be helped by federal dollars. States 
were required to spend 57% of their federal funds to serve special populations: single parents and 
homemakers were specifically named among these populations to be targeted (Weinrich, 
Weinrich & Galloway, 1988). 

The Nontraditional Employment for Women Act (NEW) passed in 1991 amended the Job 
Training and Partnership Act. Its purpose was to encourage efforts by the government to provide 
a wider range of jobs under JTPA. The NEW Act asserted that “long-term economic security of 
women is served by increasing nontraditional employment opportunities for women” (deWit, 
1994, p.l). Another federal law. School to Work was passed in 1993. This law gives planning 
grants to states to encourage businesses to coordinate opportunities for young women, and state 
plans must include strategies to ensure that the employment and training environments are free of 
sexual and racial harassment (Brustein & Mahler, 1994). 

Research Regarding Occupational Decision-Making and Choice 

Although employment equality issues began in earnest in the 1 960s with the women’s 
movement, specific studies surrounding nontraditional vocational-technical career choices for 
women are generally limited to about the last fifteen years. Studies regarding career choice and 
development fall into several general categories. Many researchers examine personality traits as 
predominant motivators, others examine socialization patterns and experiences as primary, and 
many acknowledge that perceived limitations surrounding opportunities often guide choices. 

Malloy (1993) reported that the primary influence for most nontraditional students was 
parental. Cain (1992) found that most women selected nontraditional occupations based on the 
desire to do that type of occupational work, and perceived their families and teachers to be 
supportive of their efforts. These studies confirmed the work of Psathas (1969) who included the 
values orientation of the family unit as a factor affecting a woman’s choices in life. 

In the late 1950s, John Holland (1973) developed a method of classifying personalities 
into six different categories, and proposed that occupation titles could be appropriately assigned 
to each of the types. Holland’s questions inquire about vocational interest constructs, and allow 
clients to self direct themselves to occupational options. Tokar and Swanson (1995) examined 
Holland’s categories in terms of gender. Two dimensions that identified males within Holland’s 
modes were openness and extroversion. Females in the study were identified by the strength of 
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openness, extroversion, and agreeableness factors. These findings were found to confirm 
previous research that demonstrated gender differences in terms of occupational choice. 

A major focus of research related to women and personality variables is the concept of 
psychological masculinity and femininity (Betz & Fitzgerald, 1987). Beginning in the 1970s, as 
number of researchers suggested that masculine and feminine traits must be conceptualized as 
individual dimensions; therefore, an individual of either sex could possess relatively high levels 
of both masculine and feminine characteristics. Betz and Fitzgerald noted that women’s career 
aspirations and talents are not reflected in their educational and occupational achievements as 
compm'ed to males with comparable “masculine” abilities. 

Hackett and Betz (1981), using the concept of career-related self-efficacy expectations, 
found that low career self-efficacy expectations kept women form considering traditionally male- 
dominated careers. According to these researchers, female socialization does not offer vicarious 
learning experiences or encouragement toward nontraditional pursuits and activities. 

Stringer and Duncan (1986) found that women select nontraditional, blue collar jobs for 
many of the same personal reasons that men do: the need to feel competent, the need to make a 
contribution, and the need to feel necessary and productive. They found three common factors in 
these women:“.. .women who select these jobs have notably nontraditional attitudes about sex 
roles, have fare fewer opportunities than men to enter such jobs, and they receive less societal 
support and approval” (Stringer & Duncan, 1985, p.247). 

A girl’s occupational choice is often influenced by her mother’s occupation. Daughters of 
employed mothers are more likely to have chosen to work outside of the home, and more often 
choose nontraditional occupations (Vondracek & Lemer, 1982). Fathers may also influence 
daughters’ occupational choice. Bronfenbrennen (1986) found that fathers with high 
occupational status tended to promote nontraditional jobs for oldest daughters and in families 
with no sons. 

The importance of mentors in a key element of nontraditional training (deWit, 1994). 
Pearce (1993) particularly advocated the need for mentors to guide women as they enter the 
nontraditional workplace. 

RESEARCH METHODS, PROCEDURES AND RESULTS 

The 125 subjects of this study were females enrolled in nontraditional and 
traditional postsecondary programs granting two-year degrees or certificates. Sixty eight 
respondents were enrolled in nontraditional programs; fifty seven were enrolled in traditional 
programs. Also surveyed were 26 staff members who were personally familiar with females 
enrolled in nontraditional programs. Nontraditional students and staff were from six schools 
located in Missouri, Illinois and Minnesota: traditional students were from two schools, one in 
Illinois and one in Missouri. 

Survey items were developed to answer the research questions in descriptive terms, 
including and modifying some items from surveys used in research conducted by Read (1994), 
Malloy (1993) and Cain (1993). Other questions were developed as a result of the literature 
review. An initial version of the survey instrument was field tested at a St. Louis postsecondary 
technical school in September 1994, and later reviewed by 29 educators enrolled in graduate 
programs at Southern Illinois University in November 1995. Suggestions were gathered related 
to format, vocabulary and content. Further changes were made. An additional review was later 
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conducted by 17 educators at Southern Illinois University in November 1995. The survey was 
submitted and approved by the Southern Illinois University Human Subjects Committee. 

Site visitations to postsecondary schools were made in January, February and March 
1996. Surveys were completed in an anonymous manner after a face-to-face request by the 
researcher. Surveys were completed in classrooms, technical labs, counselors’ offices and school 
cafeteria to accommodate students’ schedules. 

Fifteen nontraditional programs were represented. The highest percentages of enrollment 
were in Construction Technology (16%), Architectural Technology/CAD (12%), Refiigeration, 
Heating and Cooling /HVAC (12%), Welding (10%), Computer Technology (10%), 

Electricity /Electronics (10%), and Automotive Technology and Precision Machining at 6% each. 

Traditional students were enrolled in occupation programs including administrative 
assistant, secretarial, paralegal and office systems specialist programs. 

Age: Thirty-two percent of nontraditional occupation students were 18 to 22. The mean age of 
reentry nontraditional students was 30; the mean age of initial entry nontraditional students was 
22. In contrast, 70% of the traditional students were 18 to 22. The mean age of reentry traditional 
students was 32; mean age of initial traditional students was 19. 

Influences: The seven most influential factors, based on mean calculations of all students (both 
nontraditional and traditional occupations, younger and reentry) included type of work, high 
wages, fringe benefits, job security, experience in a class, desirable work location, and desirable 
work schedule. T test analyses comparing the mean scores of nontraditional and traditional 
occupational factors revealed no significant difference in the respondents’ ratings of the most 
influential factors of “type of work I enjoy,” and “high wages’ in regard to their chosen 
occupations. 

Funding: Students were also asked if their training was funded by an organization other than 
student loans. Eighteen (26%) of the nontraditional students recorded JTPA; JTPA was not 
recorded by any traditional students. Other funding sources for nontraditional sources included 
VA benefits/GI Bill (3), company retraining (1), and Voc Rehab (1). Funding sources for 
traditional students included employee tuition benefits (2), GI Bill (1), sports scholarship (1) and 
Civil Service Employee benefits (1). 

Mentors: Students were asked whether they have a mentor. Eighteen percent of nontraditional 
students and 23% of traditional students reported having a mentor. Twelve nontraditional 
students identified their mentors as follows: instructor/teacher (4), grandfather/father (3), friend 
(3), and counselor (2). Traditional students’ mentors included teachers (4), female relatives (5), 
counselor (1) and uncle (1). Male relatives listed as mentors had the same occupations as female 
respondents; these occupations included mechanics 92), machinist (1) and paralegal (1). 
Nontraditional Advantages and Disadvantages: The most frequently recorded factors perceived 
to be advantages of nontraditional occupations were high wages, job security and type of work. 
Sixty six (97%) of the 68 nontraditional students perceived high wages as an advantage. Job 
security was perceived as an advantage by 55 students (81%) and 55 (78%) rated type of work as 
an advantage. Desirable work schedule and work location were considered advantages by 69% of 
the respondents. Sixty three percent considered fringe benefits an advantage. Availability of 
union member ship was cited by 57% of nontraditional females as an advantage. About 49% of 
the respondents included help from a mentor and support from friends as advantages. Twenty 
two (32%) of the nontraditional respondents included parental approval as an advantage; five 
cited this factor as a disadvantage. 
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Twenty six (38%) perceived sexual harassment as a disadvantage of nontraditional work, 
and childcare was cited as a disadvantage by 10 (15%) of nontraditional respondents. 

Traditional Advantages and Disadvantages: Type of work was considered an advantage by 54 
(95%) of traditional students. Fringe benefits and parental approval were both considered 
advantages by about 80% of the traditional students. Forty four (77%) of the traditional 
respondents evaluated job security as an advantage. Thirty nine (68%) evaluated high wages as 
an advantage; eight (14%) considered wages a disadvantage. 

About one fourth of traditional respondents considered location, scheduling and sexual 
harassment as disadvantages of their chosen occupations. 

Occupational Barriers: Nontraditional students most often perceived sexual harassment (61%), 
lack of information about the occupation (54%), inadequate childcare (41%), and financial 
expense of training (32%) as barriers to entering nontraditional occupations. Approximately 70% 
of staff members at schools with nontraditional programs perceived sexual harassment, lack of 
support from family, lack of information about nontraditional occupations and inadequate 
preparation for a technical course of study as barriers. Staff also perceived lack of support from 
instructors 950%) and inadequate childcare (46%) as barriers. 

Fifty four percent of traditional students perceived sexual harassment as a barrier; 28% 
perceived the financial expense of training as a barrier. 

IMPLICATIONS AND RECOMMENDATIONS 

Predictions for the year 200 and beyond indicate that traditional employment patterns will 
generally prevail, with women concentrated in the traditionally female occupations that pay less 
than men’s jobs (Johnston and Packer, 1987). Such predictions offer challenges to educators, 
counselors, lawmakers and women who strongly believe that female movement into 
nontraditional vocational-technical occupations can bring many advantages. The differences 
between traditional and nontraditional students’ responses imply that nontraditional wage 
advantages often advocated by employment training policy makers are not so clearly perceived 
by students involved in occupation decision-making. 

Many studies indicate that women choosing nontraditional jobs are unique in personality 
and willingness to pursue careers sometimes not considered appropriate for females. Because of 
the wage gap between traditionally mens’ and women’s work, it is appropriate for educators, 
parents and counselors to provide support to all females who might benefit from nontraditional 
work. However, this enthusiasm for higher paying jobs must be balanced with the realization that 
type of work is an important factor to occupational choice. Since females often have little access 
to nontraditional occupations and work, it is imperative for educators and employers to provide 
exposure to these jobs and to present both advantages and disadvantages of nontraditional work 
to prospective students. Some staff members’ responses may indicate outdated views, especially 
in regard to occupational barriers. 

The following reconunendations are made for the purpose of guiding counselors, 
educators and women themselves in the occupational choice process as it applies to 
nontraditional occupations. 1 . Personality, aptitude and interest tests should be provided to 
females considering nontraditional work. 2. Realistic site visitations and interning opportunities 
should be provided for females considering nontraditional work. 3. Mentoring partnerships with 
women working in nontraditional occupations should be established. 4. Seminars and training 
about sexual harassment should be conducted. Practical information about how to handle tough 
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situations should be emphasized, not just the legal aspects of the issue. 5. Information about 
nontraditional courses of study should be distributed to high school counselors as well as to 
service providers in training organizations such as JTPA. 6. Females should be actively recruited 
for high school tech prep programs, allowing them to experience technical courses and activities 
prior to postsecondary programs. 7. Information and assistance in childcare should be provided 
by schools and employers for those nontraditional students requesting help. 
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Abstract 

This survey-based study examined within- and between-country differences in work-related 
values and leadership styles of U.S. and German telecommunication employee. The results 
showed country- and job level effects for cultural values and leadership styles, but only weak 
intercorrelation between the two constructs. The study points to patterns of work-related values 
different from those predicted by earlier research and identifies developmental needs in 
leadership among first line supervisors 

Despite the rapidly increasing globalization of business and industry, there is a dearth of 
cross-national and cross-cultural comparative social science research to answer the questions 
faced by organizations operating in increasingly complex and fast-changing international and 
multi-cultural environments and to provide guidance for practice. In the field of International 
Human Resource Development (IHRD) this lack is especially keenly felt (e.g.: Hansen & 
Brooks, 1994; McLean, 1991; Peterson, 1997), but also in the related fields of Human Resource 
Management (e.g.: Brewster, Tregaskis, Hegewich, & Mayne, 1996) and Organizational 
Behavior (e.g.: Lytle, Brett, Barsness, Tinsely, & Janssens, 1995). Among the reasons for to this 
paucity are the lack of global constructs and theories, the complexity of measuring country-level 
effects, and difficulties of research design. 

This comparative, cross-national, and cross-cultural study expands the body of 
knowledge of IHRD by testing a series of hypotheses related to leadership and work-related 
values of U.S. and German telecommunication employees of a global organization. The choice 
of these two countries was dictated by the strong economic, historical, political, and cultural ties 
between these two nations and the researcher’s familiarity with and interest in both nations. 
Moreover, there has been a great degree of cross-fertilization of ideas related to each other’s 
HRD-related practices in areas ranging from general and higher education, to apprenticeship 
training, labor-management relations, and work-related policies and practices. 

The study focused on three distinct research questions: (1) the degree of variance of 
cultural, work-related norms and values both within and between the two countries; (2) the level 
of variance in leadership styles and behaviors within and between the two countries; and (3) the 
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intercorrelation of work-related values and leadership styles. The research design allowed for 
three levels of analysis for each problem: (1) country-level comparisons of culture and 
leadership, comparing the two U.S. sites and the German site; (2) within-country comparisons of 
the two US sites comparing the sites located in the Mid- Western and the Eastern parts of the 
country; and (3) within-site comparisons of three levels of job categories at each of the three 
sites. The results provided a comprehensive set of answers related to the differences in work- 
related values and leadership styles and their intercorrelation by nation, by site within the U.S., 
and by job category within each site. 

Summary of Previous Research 

The concept of culture is central to the study of groups and nations and has been used in 
IHRD and international management to measure country- and group-level effects that can 
discriminate between countries and groups and thus help explain variance in the behavior or 
organizations and people. Culture has been conceptualized as a complex web of norms, values, 
assumptions, attitudes, and beliefs that are characteristic of a particular group and are reinforced 
and perpetuated through socialization, training, rewards, and sanctions (Lytle et al., 1995). 
Culture constitutes the successful attempt to adapt to the external environment, it presents the 
group’s strategy for survival (Triandis, 1994), and has been described as the “software of the 
mind” (Hofstede, 1991, p. 3). Researchers have described a wide variety of categories of 
cultural dimensions (see Lytle et al., 1995), and it is generally accepted that individuals can 
belong to any number of social groups each with its own set of norms and values (Kostova, 

1997). 

Among the most popular frameworks for studying international culture has been that of 
Geert Hofstede who published a ground-breaking study of some 1 16,000 IBM employees in 40 
countries surveyed in 1968 and 1972 (Hofstede, 1980) and used factor-analytic techniques to 
arrive at four dimensions of culture as related to work organizations: power distance, 
individualism, masculinity, and uncertainty avoidance. Hofstede later (Hofstede & Bond, 1989) 
added a fifth dimension, long-term orientation, which is a part of Confucian Dynamism. The 
framework has been applied and replicated over 60 times and the country differences predicted 
by Hofstede have been by and large confirmed (Sondergaard, 1994). 

The scholarly and practitioner literatures, by and large, have treated the country scores 
published by Hofstede (1980) from research with one company in the late 1960s and early 1970s 
as valid today and as generalizable to the entire U.S. and Germany, despite the fact that most 
replication studies were conducted in other countries. Given the single data point presented by 
Hofstede’s work and the dramatic political changes in both countries over the past 30 years, a 
replication of the original research, as conducted in this study, was warranted. 

The popularity of Hofstede’s work has further led to the oversight of possible within- 
country differences in work-related values, for instance by region or by position within an 
orgamzation. While Hofstede (1980) acknowledged the possibility that the scores of the original 
dimensions of culture were artifacts of the sample, little research has been conducted to test the 
level of homogeneity of work-related values in other samples. This study accomplished this task 
by comparing two U.S. sites to one another and each of the three sites by job category and thus 
presents an extension of the original work. 

Although several hundred definitions of leadership exist (Stogdill 1990), more recently 
the focus of research has shifted to a new class of outstanding leadership theories (House & 
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Podsakoff, 1994) that takes as its core idea the concept of charisma, defined early in this century 
by German sociologist Weber (1924/1947) as the leader’s authority based on 
“devotion... [exemplary character [and] of the normative patterns or order revealed by him (sic)” 
(p. 328). Based on this notion, Bass (1985) and colleagues developed a theory of 
transformational leadership, distinguishing between transformational leadership (and its 
components charisma, intellectual stimulation, individual consideration, and inspirational 
motivation); transactional leadership (contingent reward and management-by-exception); and 
laissez-faire (or non-) leadership. Transformational leadership is said to exert an augmentation 
effect, that is to add to the levels of productivity, satisfaction, and effectiveness associated with 
transformational leadership alone (Avolio, Bass, and Jung, 1995). The factor structure of the 
theory has been confirmed in many replications and the instrument refined and translated into a 
number of languages. 

None of the replications, however, addressed possible differences in leadership style by 
site, by population within site, or by country. Focusing on single-country, single-site research 
settings, the theory has not been tested in light of possible differences in leadership styles by 
level within the organization or by region, and scant attention has been paid to differences in 
transformational leadership style by country. This study accomplished this by testing for 
differences in leadership styles within and between sites and countries. 

A third purpose for the study was to empirically test whether specific work-related values 
were related to specific leadership styles. While the topic of culture and leadership has received 
much theoretical attention in the literature (e.g.: Hofstede, 1980; Schein, 1985; Triandis, 1994), 
little empirical research exists relating specific cultural values to specific styles of 
transformational or transactional leadership. 

Research Setting, Design, and Methodology 

The study was conducted using a causal-comparative, ex-post-facto, one-shot survey 
design. The population consisted of the 5,400 employees at three manufacturing sites of a 
Fortune 500 multi-national telecorrummication organization headquartered in the U.S. The three 
sites were located in Ohio (OH), New Jersey (NJ), and Niimberg/Germany (NU) and reported to 
the same U.S.- based Vice President of Operations. All three sites were involved in 
manufacturing telecommunication transmission equipment, had like technologies and work 
processes, and identical work policies and procedures. The sites differed in size and age, the OH 
site was the largest and oldest, the NJ site the smallest and yoimgest. In consultation with 
representatives at the three sites, a survey was administered to a stratified, non-proportional, 
random sample of the population, using Hofstede’s 1994 version of the Value Sirrvey Module, 
VSM 94 (Hofstede, n.d.), Bass and Avolio’s Multifactor Leadership Questionnaire, MLQ5x 
(Avolio, Bass, and Jimg, 1995), and a series of standard demographic questions. Both the VSM 
94 and the MLQ5x have been used extensively in the past and have known psychometric 
properties; the instruments were obtained from the authors in English and German and given to 
the participants in their native language. The survey was administered to 3,540 employees (66% 
of population) and yielded an overall response rate of 53% (Table 1). 



O 

ERIC 



125 



136 



Table 1 



Sample Sizes. Responses and Response Rates by Job Category and Location 



Site/Job 


Ohio 




New Jersey 


Niimberg 


Total 




Category 






V 














Initial 


Res- 


Initial 


Res- 


Initial 


Res- 


Initial 


Res- 




Sampl 


ponse 


Sampl 


ponse 


Sampl 


ponse 


Sampl 


ponse 




e 




e 




e 




e 




Production 


939 


196 


255 


122 


745 


528 


1,939 


846 


Employees 




(21%) 




(48%) 




(71%) 




(44%) 


Engineers 


305 


131 


134 


34 


800 


468 


1,239 


633 






(43%) 




(25%) 




(59%) 




(51%) 


Managers 


186 


105 


23 


6 


150 


53 


359 


164 






(56%) 




(26%) 




(35%) 




(46%) 


Invalid/No 




6 




25 




198 




229 


Answers 




(<1%) 




(6%) 




(12%) 




(6%) 


Total 


1,430 


438 


412 


187 


1,695 


1247 


3,537 


1872 






(31%) 




(45%) 




(74%) 




(53%) 



Note : Percentages based on Initial Sample Size Row Totals 



The error limit (Wunsch, 1986) associated with this response was less than plus/minus 3 
percent overall and ranged from plus/minus 2% to plus/minus 10% for specific subpopulations. 
Chi-square tests of demographic information of respondents with information obtained from the 
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sites’ personnel information system showed that survey respondents were representative of the 
plant population with the exception of the overrepresentation of women in the sample. In 
addition to the survey, a series of open-ended interviews were conducted at each site to obtain 
contextual information. The data were analyzed and feedback sessions conducted at each of the 
three sites with managers, engineers, and production employees. The feedback sessions resulted 
in action plans at each site to improve leadership styles and collaboration within and among the 
three sites. 



Results 

Demographic information collected from the 1,872 survey respondents revealed 
differences by site and by job category for gender, age, level of education, and union 
membership (Table 2). With the exception of production employees at the Ohio plant, the 
majority of employees in all job categories were male; employees at the Ohio plant were older 
and had lower levels of education for production and engineering positions than at the other two 
plants. The New Jersey and Niimberg plants were similar in their demographic profile, except 
for the fact that the German production employees were unionized. 

The scores for leadership styles and work-related values were calculated based on the 
formulae provided by the test authors; test statistics for the leadership scales revealed ranges of 
the mean scores and standard deviations similar to those attained in previous studies, with 
measures of internal consistency (Chronbach’s alpha) ranging from a = .68 to a = .83, meeting 
the generally expected level of a = .7. Means and standard deviations for the culture 
dimensions varied from previously published studies; internal reliability was a = .83 for the 
entire instrument, but below the .70 mark for Power Distance and Uncertainty Avoidance. These 
low reliabilities pose questions about the internal reliability and factor structure of these two 
dimensions and pose a limitation to this study. As predicted by the theory, the four 
transformational leadership scales were highly and positively intercorrelated, while transactional 
and the non-leadership factors were negatively correlated with transformation leadership. There 
were only marginal (r < .2) correlations between the cultural dimensions, suggesting the 
independence of the dimensions from one another. 
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Table 3 

Chronbach Alphas and Pearson’s Product-Momet Correlations f p < .05 onlvl of Leadership and Culture Dimensions (N=1.8721 
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Differences in Cultural Values and Leadership Styles 

Because of main effects of demographic variables on leadership and culture, they were 
statistically controlled for in the Analyses of Covariance (ANCOVA) models used to test for 
differences vsdthin, among, and between sites. Each ANCOVA was conducted three times: (1) 
by merging the two U.S. sites and comparing them to the German site to test for country effects; 

(2) by comparing the New Jersey and Ohio sites to test for regional, within-country effects; and 

(3) for each of the three sites by job category to test for job category effects. Follow-up pairwise 
tests were conducted using the Bonferroni procedure. 

, The study found country-level effects for three of the five culture dimensions: Power 
Distance (Germany higher). Individualism (U.S. higher), and Masculinity (U.S. higher). There 
were no differences in the levels of Uncertainty Avoidance or Long-term Orientation (Table 4). 
These findings depart from the results reported in Hofstede’s (1980) original research where the 
U.S. sample had scored higher in Power Distance and where the differences in the two countries’ 
scores for Masculinity and Uncertainty Avoidance had been much greater. The country-level 
analyses also showed that production employees and engineers in the two countries differed, 
while U.S. and German managers’ scores did not. ~ 
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Table 4 



Analysis of Covariance for Culture Dimensions rN=1.892): Hofstede’s ri980) Scores in 
(Brackets') 



Dimension 


U.S. 


M 

Germany 


F 


Power Distance 


29.05 


35.21 


13.10*** 




(40) 


X35) 




Individualism 


79.07 


53.18 


165.2*** 




(91) 


(67) 




Masculinity 


10,51 


-19.96 


52.05*** 




(62) 


(66) 




Uncertainty Avoidance 


102.55 


105.06 


2.15 ns 




(46) 


(65) 




Long-term Orientation 


43.00 


43.84 


.07 ns 




(29) 


(31) 





Levels of Significance: .001; .01; * .05 



When comparing culture and leadership scores of the two U.S. sites, testing hypotheses 
related to regional (within the U.S.) differences, the results showed differences in Power 
Distance (Ohio site higher) for production employees and engineers. None of the other culture 
dimension scores differed in the ANCO'VA model. 

The study also tested for within-site differences in work-related values. Here, the results 
differed by site: At the New Jersey plant, no differences were found for any of the five culture 
dimensions among managers, engineers, and production employees. At the Ohio site, managers 
scored lower on Power Distance and Uncertainty Avoidance than the rest of the organization, 
while production employees scored the lowest on individualism (Table 5). 
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Comparison of Within-Site Difference in Culture Dimensions: Ohio rN=438') 



M F 


Pairwise 


Dimen- Product. Engineers Managers 


Compari- 


sion 


sons 



PDI 


43.73 


36.40 


19.55 


5.86** 


E>M;P>M 


IND 


66.14 


88.35 


93.17 


67.07*** 


E>P;M>P 


MAS 


3.41 


10.87 


1.29 


.47 ns 


- 


UAI 


118.94 


102.32 


78.71 


8.67*** 


E>M;P>M 


LTO 


43.48 


46.68 


47.10 


.06 ns 


- 



Ppi: Power Distance, IND: Individualism; MAS: Masculinity; UAI: Uncertainty Avoidance; LTD: Long-term 
Orientation; Significance Levels: *** .001; ** .01; * .05 
P: Production Employees, E: Engineers, M: Managers 

At the German site (Table 6) , managers also scored lower on the Uncertainty Avoidance 
scale, and there were differences in the level of Individualism (production employees lower than 
engineers), Masculinity (production employees higher than engineers), and Long-term 
Orientation (managers higher than production staff). 
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Table 6 

Comparisons of Within-Site Culture Dimensions by Job Category: Ntimberg rN=1.247') 



M F Pairwise 



Dimen- Product. Engineers Managers Compari- 

sion sons 



PDI 


34.49 


33.53 


36.37 


.04 ns 


— 


IND 


46.44 


61.94 


56.15 


4.68*** 


E>P 


MAS 


-10.61 


-34.20 


-14.71 


6.48** 


P>E 


UAI 


100.68 


108.45 


76.40 


8.51*** 


P>M; E>M 


LTO 


40.68 


45.80 


60.67 


4.04** 


M>P 



PDI: Power Distance, IND: Individualism; MAS: Masculinity; UAI: Uncertainty Avoidance; LTO: Long-term 
Orientation; Significance Levels: *** .001;** .01; * .05 
P: Production Employees, E: Engineers, M: Managers 

Coimtry-level effects were also found for the transformational leadership dimensions of 
Charisma and Inspirational Motivation (U.S. higher), as well as for Intellectual Stimulation 
(Germany higher). These differences were found among production employees and engineers, 
but not among mangers. 

When comparing leadership styles of the two U.S. sites, the only differences were 
between NJ engineers who scored higher on Management-by-Exception than their counterparts 
at the OH site, and NJ managers who scored higher on Management-by-Excpetion and Laissez- 
Faire Leadership than their OH peers. 

With respect to leadership, the NJ site yielded a homogeneous picture and no differences 
were found among the three employee groups. At the Ohio (Table 7) and the German (Table 8) 
sites, managers scored higher on transformational leadership styles than did engineers and 
production employees. 
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Table 7 



Comparison of Within-Site Differences in Leadership: Ohio (N=432) 





M 

Product. Engineer Managers 


F 


Pairwise 

Compari- 

sons 


TFL 


2.22 


2.37 


2.44 


4.15* 


M>P 


CHA 


2.31 


2.55 


2.72 


3.25* 


M>P 


MOT 


2.54 


2.72 


2.79 


.81 ns 




STM 


2.05 


2.18 


2.62 


8.36*** 


M>P;M>E 


CONS 


1.99 


2.17 


2.44 


3.83* 


M>P 


TAL 


1.79 


1.63 


1.67 


1.08 ns 




REW 


2.22 


2.24 


2.44 


1.32 ns 




MBE 


1.57 


1.32 


1.30 


2.67 ns 




LFL 


1.06 


.87 


.76 


2.35 ns 





Significance Levels: *** .001; ** .01; * .05 

TFL: Transfonnational Leadership; CHA: Charisma; MOT: Inspirational Motivation 

STM: Intellectual Stimulation; CONS: Individual Consideration; TAL: Transactional Leadership; REW: Contingent 
Reward; MBE: Management-by-Exception; LFL: Laissez-faire 
Leadership; P: Production Employees, E: Engineers, M: Managers. 
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Table 8 



Comparison of Within-Site Differences in Leadership: Niimberg rN=1.247') 





M 






F 


Pairwise 




Product. 


Engineers 


Managers 




Compariso 




Employ. 








ns 


TFL 


2.31 


2.14 


2.39 


3.32* . 


M>E 


CHA 


2.31 


2.05 


2.40 


4.61* 


P>E; M>E 


MOT 


2.40 


2.11 


2.42 


5.63** 


P>E; M>E 


STM 


2.40 


2.26 


2.43 


.99 ns 


— 


CONS 


2.21 


2.11 


2.26 


.89 ns 


— 


TAL 


1.73 


1.64 


1.65 


.40 ns 


— 


REW 


2.32 


2.07 


2.27 


1.15 ns 


— 


MBE 


1.53 


1.48 


1.39 


1.32 ns 


— 


LFL 


.97 


1.05 


1.01 


.06 ns 




Significance Levels: *** 


.001; ** .01; * 


05 







TFL: Transformational Leadership; CHA: Charisma; MOT: Inspirational Motivation 

STM: Intellectual Stimulation; CONS: Individual Consideration; TAL: Transactional Leadership; REW: Contingent 
Reward; MBE: Management-by-Exception; LFL: Laissez-faire Leadership; P: Production Employees, E: Engineers, 
M: Managers. 
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Correlational Analyses of Culture and Leadership 



Partial correlation analyses, controlling for demographic variables, were conducted to 
measure the relationship between cultural dimensions and leadership. As before, these analyses 
were conducted at the country, regional, and site levels. Overall, the analyses failed to show 
significant relationships between the two constructs. Where significant, partial correlation 
coefficients rarely exceeded r = .2, suggesting spurious relationships and artifacts of the sample 
size. Low- to medium correlations were, however, found for the U.S. engineers (both U.S. sites 
combined) where Power Distance and transformational leadership dimensions were positively 
correlated (r’s ranging from .3 1 to .41). Correlations of similar size were also found at the 
German plant managers for the relationships between Individualism and Transactional Reward 
and Intellectual Stimulation and Long-term Orientation. 

These results show a generally limited intercorrelation between culture and leadership, 
except for U.S. engineers and German managers. 

Implications for Practice and Research 

This study has implications for practice at two levels. For the organization, it yielded a 
fine-grained analysis of work-related values and leadership styles. As the three plants are 
moving toward closer cooperation with one another, these measurements of culture and 
leadership gave voice to areas of similarities and differences, resulted in action planning to 
reduce the high levels of uncertainty avoidance and to raise the use of transformational styles of 
leadership, and set a baseline for measuring progress over time. 

Although the study did not generalize beyond the walls of the three sites, it has 
implications for HRD professionals working within the U.S. and internationally. First, as this 
study shows, work-related values differ by country, by region, and by level within the ' 
organization. Further, these differences change over time. The study of culture is in its infancy; 
assuming that Hofstede’s results, or those of any other cross-cultural study (including this one) 
can apply to an untested population appears, therefore, as a dangerous and unwarranted 
overgeneralization. In working with different groups, whether nationally or internationally, 
where work related values are of importance, HRD professionals are well advised to conduct 
measurements like the ones used in this study in order to tailor the intervention to their specific 
cultures. A similar argument appears warranted for expatriate training: given the amount of 
within-country variance of culture, an assessment of the specific work-related values of the host 
culture will provide the most valid and reliable information. 

This study found a predominance of transactional leadership styles among production 
employees. Given the clear link between transformational leadership and increased performance 
and quality of work life, this finding, if home out in other settings, suggests a developmental 
agenda for first-line supervisors, team leaders, and those in charge of production employees who, 
historically, have received the least amount of formal training and development. 

The study also has clear implications for theory and points to future research needs. In 
this study, work-related values were neither uniformly distributed nor shared within the two . 
countries, raising questions about the utility of the construct of national culture in multi-cultural 
societies like the U.S. and Germany. Further, one of Hofstede’s measures returned unacceptably 
low internal reliability measures, calling into questions the validity of the underlying construct. 

Related to leadership theory, the study suggests the existence of heterogeneity of 
leadership styles within the same organization, with clearly lower levels of transformational 
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styles among German employees and among U.S. and German production workers. Finally, the 
moderate-size correlation between transformational leadership and high power distance among 
German engineers suggests a possible limitation to the extent to which transformational 
leadership can be implemented in that group. Despite the general trend toward more 
participatory and egalitarian working relationships, leadership through a strong vision and 
identification with the leaders might not be accepted in all situations. 

The agenda for future research suggested by this study focuses on the need to more 
clearly define the culture construct and develop reliable and valid measures, to better understand 
the limitation that work-related values might place on transformational leadership, and to 
investigate the reasons for the low levels of transformational leadership among production 
employees. 
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Abstract 

This descriptive study was designed to describe instructors ' use of assessment 
methods in post-secondary technical and occupational classrooms and laboratories. 

Data were collected from a random sample of all post-secondary technical and 
occupational instructors within the State of Ohio. Results indicated instructors tend to 
use information from paper and pencil test the most in making instructional decisions. 
Instructors used information from performance assessments the next most often, which 
was followed by informal observations. Post-secondary instructors tended to rarely use 
information from standardized tests, portfolios, and essay methods. Instructors neither 
agreed or disagreed that they faced potential constraints while conducting assessment 
activities. More than four-fifths of the instructors reported a positive attitude toward 
assessment and more than one-half considered themselves to be very competent in 
assessment. It was found that instructors within the program area of marketing 
accounted for more than six percent of the variation of using portfolios in making 
instructional decisions. Two program areas (marketing and business occupations) each 
accounted for three percent of the variation in the use of essay methods in making 
instructional decisions. Instructors ’ attitudes toward assessment was found to explain 
11% of the variation in instructors use of performance assessment and four percent of the 
variation in instructors ’ use of portfolios. The perceived competency level of instructors 
in assessment contributed about three percent of the variation in instructors ' use of 
portfolios. 
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Introduction 



Since the 1970s the public, elected officials, state agencies, and the media have 
become more interested in education. This interest has been due in part to the 
achievement of our students, which does not compare well with counterparts in other 
countries. Hudgins (1993) stated that although costs have been rising for higher 
education, student achievement has flattened out and sometimes even declined. Many 
individuals in post-secondary education paid little attention to the debate caused by this 
report and assumed that this was a problem of K-12 education. The public, however, did 
not see the separation between post-secondary and K-12 education and looked at this as 
one educational system (Hudgins, 1993). Because of this lack of separation, there has 
been an increased interest and sometimes demand for educational institutions to show 
greater accountability to suggest how much and what students are learning at all levels of 
education. Post-secondary institutions have not shown to legislators, or to the public, a 
strong and obvious correlation between investment and return (Hudgins). 

Colleges and universities know how smart their students were when these students 
arrived on their campuses, but they do not know very much about what they have learned 
while they were there (Bok, 1992). Because there has been little investigation on how 
well post-secondary instructors teach or how much their students learn, deciding the 
effectiveness of their programs has been difficult. This presents a problem when trying to 
convince the public that education is a top priority of these institutions. Since this focus 
has not shifted, colleges and universities have continued to be vulnerable to the attacks on 
their curricula, faculty, tuition, and other issues (Bok, 1992). 

Statement of the Problem 



How technical and occupational instructors use assessment information in the 
classroom and laboratory and whether its use is effective, can play a major role in 
enhancing and documenting both instruction and learning. The literature revealed very 
little about assessment practices and their use by post-secondary instructors. Recent 
research conducted on the quality and effectiveness of general-education instructor 
training in measurement and assessment has implied that these individuals may not be 
acquiring the appropriate skills necessary to use assessment skills effectively. If this is 
true for general education instructors, what are the assessment skills needed by technical 
and occupational instructors? Information on the assessment skills needed and the use of 
assessment information by technical and occupational education instructors is not 
available. 



Purpose and Objectives of the Study 

The purpose of this study was to describe assessment activities, problems, 
attitudes, and perceptions of Ohio post-secondary technical and occupational education 
instructors. The specific objectives of this study were to: 



1 . Describe Ohio post-secondary technical and occupational education instructors in 
terms of the following demographic characteristics: age, sex, related work 
experience, teaching experience, program area taught, and educational level. 

2. Describe Ohio post-secondary technical and occupational education instructors' 
perceptions of the use of student assessment information in making instructional 
decisions. 

3. Describe how Ohio post-secondary technical and occupational education 
instructors face potential constraints while conducting assessment activities. 

4. Describe Ohio post-secondary technical and occupational education instructors’ 
attitudes toward the assessment process. 

5. Describe Ohio post-secondary technical and occupational education instructors' 
perceptions of their level of competence in the assessment process. 

6. Examine the relationships between Ohio post-secondary technical and 
occupational education instructors’ use of assessment information and: 

a. selected demographic characteristics, 

b. perceived level of competence in the assessment process, 

c. their attitudes toward assessment, and 

d. constraints to the assessment processes. 

7. Determine the proportion of variance in Ohio post-secondary technical and 
occupational education instructors' perceived use of assessment information in 
instructional decision-making that can be explained by the independent variables 
of attitudes toward assessment, competence in the assessment process, constraints 
to the assessment process, and selected demographic characteristics. 

METHODOLOGY 

The methodology used to conduct the research is described in this section. The 
methodology is organized and presented in the following sections: (a) research design, 
(b) population and sampling, (c) instrumentation, (d) data collection, and (e) data 
analysis. 



Research Design 



This descriptive-correlational study was designed to explore and describe the 
assessment practices of Ohio post-secondary technical and occupational instructors. The 
nature and strength of relationships between assessment use and instructor attitudes, 
instructor competence in assessment, constraints to assessment, and instructor 
characteristics were examined. The dependent variable in this study was instructor use of 
student assessment information in making instructional decisions. Independent variables 
included instructor attitudes toward assessment, instructors’ competence in assessment, 
and potential constraints faced by instructors while conducting assessment activities. 




140 



152 



Population and Sampling 



The target population for this study was all full-time post-secondary technical 
instructors in the State of Ohio during the spring of 1996. A random sample of 3 1 8 
individuals was drawn from the population as recommended by Krejcie and Morgan 
(1970) with a 95% confidence level of obtaining a representative sample of the 
population. 



Instrumentation 

The five-part survey used for this research was adapted from a survey used in a 
similar research study (Kershaw, 1993). The dependent variable, instructor use of 
assessment information, was measured in Part I of the instrument. Instructors were asked 
to indicate the extent to which they use information from six types of assessment methods 
in addressing nine different instructional decisions. The six types of assessment methods 
used in the study included: objective paper and pencil methods, standardized test scores, 
performance assessments, informal observations, portfolios, and essay methods. The 
participants in this study were asked to indicate the extent to which they use results from 
each of the six assessment methods in making nine different types of instructional 
decisions. The instructional decisions addressed were: planning for instruction, 
diagnosing student weaknesses, monitoring student progress toward course objectives, 
motivating students to learn, evaluating the effectiveness of instruction, evaluating the 
instructional materials used, encouraging students to assess their own work, assigning 
grades, and grouping students for instructional activities. A five-point Likert-type scale 
ranging from “never use” to “always use” was used. 

Part II of the questionnaire was designed for measuring the first independent 
variable, competence in assessment. The respondents were asked to indicate the 
perceived level of their competence in the assessment process. The competency 
statements used for this section were based upon “Standards for Teacher Competence in 
Educational Assessment of Students” (American Federation of Teachers, National 
Council on Measurement in Education, & National Education Association, 1990). A 
five-point Likert-type scale was used which ranged from “not competent” to “extremely 
competent.” 

Instructors’ attitudes toward assessment were measured in Part III of the 
instrument using a semantic differential scale. This scale consisted of eight bi-polar 
adjectives that represent the concept of assessment. A seven-point scale was used for 
each pair of adjectives. Respondents were asked to place a check mark between the 
adjectives at the point that best indicated how they would describe their attitude toward 
the overall assessment process. 

Part IV of the instrument was designed to measure the perceived constraints that 
instructors may face during their assessment activities. A five-point Likert-type scale 
was used that ranged from “strongly disagree” to “strongly agree”. Participants were 
asked to indicate their level of agreement or disagreement with nine constraint 
statements. 
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The personal characteristics of each instructor were measured in Part V of the 
questionnaire. Respondents were asked to provide personal information regarding the 
characteristics of sex, teaching experience, related work experience, teaching area, level 
of education and present age. 

A panel of experts was used to establish content and face validity for the survey 
instrument. The panel of experts was used to ensure that the instrument had clearly 
defined statements and questions, made suggested changes to statements and questions, 
offered suggestions for the addition or deletion of statements and questions, and made 
comments on the format and general appearance of the survey instrument. 

After the instrument was revised, it was pilot tested for reliability using a group of 
respondents from a community college outside the State of Ohio. A test-retest method 
was also used to test the reliability of the instrument over a period of time. The time 
interval between the test and retest was two weeks. The results from test #1 and test #2 
were compared for percent agreement with values that ranged from .85 to .92 for each 
item. Measures of internal consistency were calculated from the data obtained from the 
first test of the test/retest method. Cronsbach’s Alpha ranged from .77 to .94 for each of 
the six sections of Part I, .85 for the 21 summed items in Part II, .81 for the semantic 
differential in Part III, and .54 for the sum of the nine items in Part IV. 

Data Collection 



The data for this study were collected by a mail questionnaire. This data collection 
consisted of an initial mailing of questionnaires being sent out and approximately one 
week later a post card reminder was sent to all participants in the study. This post card 
reminded participants of the importance of the study and the importance of each response. 
It further thanked the respondents and asked the non-respondents to take the time to 
complete and return the questionnaire. Approximately two weeks after the first mailing a 
second packet was mailed to all non-respondents. The final correspondence with the 
participants included a second post card reminder sent to all non-respondents 
approximately one week after the second mailing was sent. Questiormaires were received 
from 170 instructors that represented an overall response rate of 53%. Of the 170 surveys 
returned, 93% (158) were useable. This represented an overall useable response rate of 
slightly less than 50%. 

A 10% random sample of non-respondents was selected and contacted by 
telephone. However, to obtain 10% responses, it was necessary to contact more than 
20% of the non-respondents. Differences between non-respondents and respondents on 
each part of the questiormaire were compared using a t-test. The significance level of 
alpha for this test was set at .05. No significant differences were found between the two 
groups for each part of the questiormaire. However, an over sampling of the non- 
respondents was conducted to obtain this information, therefore the results of this study 
carmot be generalized beyond the respondents. 
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Data Analysis 



Descriptive statistics provided measures of central tendency and variability for 
organizing, summarizing and analyzing differences and likenesses between groups. 
Pearson’s r coefficient, point bi-serial correlation coefficients, and Cramer’s V were used 
to summarize the magnitude and direction of the relationship between variables. The 
Davis (1971) conventions were used to describe the measures of association (Table 1). 

Table 1 

Conventions Used to Describe Measures of Association 



O 

ERIC 



Coefficient Description 

.70 or higher Very Strong Association 

.50 to .69 Substantial Association 

.30 to .49 Moderate Association 

. 1 0 to .29 Low Association 

.01 to .09 Negligible Association 

Source: Davis, (1971) 

Semi-partial multiple regression was used to calculate the unique relationships 
between independent variable sets and the dependent variable, use of assessment 
information in making instructional decisions. The stepwise entry method of independent 
variables into the multiple regression models was used based on the exploratory nature of 
the study, the fact that interval, ordinal, and nominal variables were investigated, and the 
lack of theory to guide the order in which variables were entered into the equation. 

FINDINGS 

This section reports findings related to the following: (a) characteristics of the 
respondents, (b) instructors’ use of assessment information, (c) instructors’ perceived 
competence in assessment, (d) instructors’ attitudes toward assessment, (e) instructor 
perceived constraints to assessment, (f) relationships between dependent and independent 
variables, and (g) multiple regression models. 

Demographic and Background Characteristics of the Sample 

This study examined six demographic and background characteristics for the 
sample that included age, sex, years of teaching experience, years of work experience, 
program area, and level of education. From the variable of age it was found that 44.2% 
of the sample was in an age group that ranged from 40-49 years (Table 2). Slightly more 
than 27% of the respondents were between 50-59, and 16.2% were between 30-39 years 
of age. Instructors ranged in age from 25 to 67 with a mean age of 47.03. Gender data 
revealed that females comprised 57% and males 43% of the sample (Table 2). 
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Table 2 



Age and Sex of Instructors (n=158) 



Age 



Sex 



Charactersitics 


f 


% 


Under 30 


25 


16.2 


30-39 


25 


16.2 


40-49 


68 


44.2 


50-59 


42 


27.3 


60 and over 


16 


10.4 


Total 


154“ 


100.0 


Female 


90 


57.0 


Male 


68 


43.0 


Total 


158 


100.0 



Mean = 47.03 
SD = 9.4 



Tnformation was missing for four respondents on age. 

Years of teaching experience ranged from one to 38 years with the mean being 15 
years of experience (Table 3). The largest group of instructors (23.1%) taught for 16-20 
years while the smallest group (2.6%) taught for more than 3 1 years. In terms of related 
work experience, 12.3% had 16-20 years of related work experience before or concurrent 
to teaching, 18.8% had 1 1-15 years, and 24.7% had 6-10 years (Table 3). Slightly more 
than 34% had 0-5 years of related work experience. 
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Table 3 



Instructors’ Years of Related Work and Teaching Experience (n = 158) 



Characteristics 


f 


% 


Years of Related Work Experience 






0-5 


53 


34.4 


6-10 


38 


.24.7 


11-15 


29 


18.8 


16-20 


19 


12.3 


21-25 


5 


3.3 


26-30 


6 


3.9 


31-35 


4 


' 2.6 


Total 


154“ 


100.1 



Years of Teaching Experience 



1-5 


20 


12.8 


6-10 


32,. 


20.5 


11-15 


27 


17.3 


16-20 


36 


23.1 


21-25 


19 


12.2 


26-30 


18 


11.5 


31 or more 


4 


2.6 



Mean = 10.96 
SD = 7.92 



Mean = 15.49 
SD = 8.34 



Total 



156*’ 100.0 



“Information was not available from four respondents on years of related work 
experience. '’Information was not available from two respondents on years of teaching 
experience. 

Health occupations comprised the largest group in the study with 38.5% of all 
instructors (Table 4). Business occupations also consisted of a large portion of 
instructors within the study (26.3%), and was followed by engineering at 12.8%. 
Agriculture, family and consumer sciences and human services contributed less than two 
percent each. Four-fifths of the respondents had completed a graduate degree (80%) 
(Table 5). Slightly less than 10% had completed some graduate work, and less than six . 
percent had a bachelors degree. 
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Table 4 



Program Area of Instructors (n=158) 



Program Area 


f 


% 


Agriculture 


3 


1.9 


Business Occupations 


41 


26.3 


Marketing 


5 


3.2 


Family and Consumer Sciences 


2 


1.3 


Trade and Industrial 


10 


6.4 


Health Occupations 


60 


38.5 


Engineering 


20 


12.8 


Human Services 


2 


1.3 


Other 


13 


8.3 


Total 


156“ 


100.0 



“Information was not available from two respondents. 
Table 5 

Instructors Level of Education (n=l 58) 



Level of Education 


f 


% 


Associates Degree 


5 


3.2 


Bachelors Degree 


9 


5.8 


Some Graduate Work 


15 


9.7 


Masters Degree 


106 


68.4 


Doctorate 


18 


11.6 


Other 


2 


1.3 


Total 


155“ 


100.0 



“Information was not available from three respondents. 
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Use of Assessment Information 



Table 6 displays the results of instructors’ use of assessment information. 
Instructors rated assessment information provided from objective paper and pencil 
methods (M=3.92) as more useful in making instructional decisions than information 
from the other five assessment methods. Performance assessment was rated second 
highest by instructors for providing information in making decisions (M=3.78). 
Information derived from informal observations was rated third (M=3.64), essay items 
were fourth (M=2.49), and standardized test scores rated fifth (M=1.90). Portfolios 
provided information deemed of the least use by instructor in making educational 
decisions (M=1.78). 
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Use of Assessment Information Generated From Six Assessment Methods (n=158) 
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Potential Constraints to Assessment 



Table 7 depicts the summed data gathered from instructors on potential 
constraints they faced while conducting assessment activities. Over one-half (55.1%) of 
the instructors neither agreed nor disagreed that they faced potential constraints while 
conducting their assessment activities. Slightly more than 42% of instructors agreed that 
they faced potential constraints when conducting assessment activities, while slightly 
more than one percent disagreed potential constraints affected their assessment activities. 

Table 7 



Overall Constraints to Assessment Scores (n = 1 58) 



Scale Value 


Constraint Score 


f 


% 


Strongly Disagree 


9-13 


0 


0.0 


Disagree 


14-22 


2 


1.27 


Neither Agree or Disagree 


23-31 


87 


55.05 


Agree 


32-40 


67 


42.41 


Strongly Agree 


41 -45 


2 


1.27 


Note. Missing data was recoded to the mean. 







Mean = 30.82 
SD = 3.97 



Instructors’ Attitude Toward Assessment 



Attitudinal measure of post-secondary instructors was made by summing the eight 
responses on the semantic differential scale. Post-secondary instructors reported having a 
more positive (81.1%) than negative (5%) attitude toward assessments (Table 8). The 
remaining instructors (13.9%) in this study reported being neutral in their attitude toward 
assessment. 

Table 8 



Instructors’ Summated Attitude Scores Towards Assessment (n = 158) 



Summed Attitude Scores 


f 


% 


8-11 


0 


0.0 


12-19 


1 


0.6 


20-27 


7 


4.4 


28-35 


22 


13.9 


36-43 


47 


29.8 


44-52 


62 


39.2 


53-56 


19 


12.1 


Note. Missing data was recoded to the mean. 





Mean = 42.65 
SD = 8.19 




149 



162 



Perceived Level of Competence in Assessment 



A measure of perceived competence in assessment was calculated by summing 
post-secondary instructor responses to 21 competency statements (Table 9). The summed 
scores across the 2 1 statements indicated that slightly less than six percent of technical 
and occupational instructors considered themselves to be extremely competent in the 
overall assessment process, 51.9% to be very competent, and 42.4% to be moderately 
competent. None of these instructors considered themselves to be slightly competent or 
not competent in assessment. 

Table 9 

Summed Instructor Competency Scores (n = 158) 



Scale Value 


Summed Score 


f 


% 




Not Competent 


21-31 


0 


0.0 


Mean = 76.5 


Slightly Competent 


32-52 


0 


0.0 


SD = 10.76 


Moderately Competent 


53-73 


67 


42.4 




Very Competent 


74-94 


82 


51.9 




Extremely Competent 


95 - 105 


9 


5.7 





Note. Missing data was recoded to the mean. 



Relationships Between Dependent and Independent Variables 

Potential constraints faced by instructors when conducting assessment activities 
was found to have a low association with the use of standardized test scores (r=.25) and 
performance assessment (r=.20) (Table 10). Attitude toward assessment had a moderate 
positive association (r=.33) with the use of performance assessment. Program area had a 
substantial association (r^=.58) with the use of essays as an assessment method, and 
competence in assessment had a low positive association (r=.18) with instructor use of 
information from portfolios as an assessment method. 
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Intercorrelations Between Independent Variables and Assessment Methods (n=158) 
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Note. Unless otherwise noted reported coefficients are Pearson product-moment correlations. 
Relationships among dichotomous and multichotomous nominal variables are reported as 
Cramer’s V correlations. Relationships among dichotomous nominal and interval variables are 
reported as Point-biserial correlations. Relationships among ordinal and interval variables are 
reported as Spearman rank-correlations. Relationships among multichotomous nominal and 
ordinal variables are reported as Cramer’s V correlations. Relationships among multichotomous 
nominal and interval variables are reported as Cramer’s V correlations. 

“Multichotomous nominal variable. 

‘’Dichotomous nominal variable. 

“Ordinal variable. 

* P<.05 
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Regression of Dependent Variable Upon the Independent Variable 



It was found that instructors within different program areas placed more weight upon the 
use of portfolios as an assessment tool (Table 1 1). The program area of marketing accounted for 
more than six percent of the variation of using portfolios in making instructional decisions. Two 
program areas (marketing and business occupations) each accounted for slightly more than three 
percent of the variation in the use of essay methods in making instructional decisions (Table 12). 

Table 1 1 

Semi-Partial Regression of Use of Portfolios On All Independent Variables (n = 1 58)(Stepwise 
Entry) 



Variables 


sR' 


b 


t 


D 


Marketing 

(Program Area)* 


.065 


1.53 


3.49 


<.001 


Potential Constraints 


.029 


-.581 


-3.30 


<.001 


Attitude 


.043 


.184 


2.38 


.018 


Competence 


.033 


.367 


2.46 


.015 



(Constant) 1 .42 



Standard error = .935 
R'=.17 

Adjusted R^= .15 

For model: F = 7.37, p <.001 

“Multichotomous variables were dummy coded: Marketing = 1 and Other = 0 
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Table 12 



Semi-Partial Regression of Use of Essay Methods On All Independent Variables (n = 
158)(Stepwise Entry) 



Variables 


sR^ 


b 


t 


P 


Marketing 

(Program Area)® 


.033 


1.26 


2.51 


.013 


Business Occupations 
(Program Area)® 


.031 


.448 


2.23 


.027 


(Constant) 


2.35 









Standard error = 1.077 
R^=.064 

Adjusted R^= .052 

For model: F = 5.08, p <.001 

“Multichotomous variables were dummy coded: Marketing = 1 and Other = 0. Business 
Occupations = 1 and Other = 0 

Instructors’ attitudes toward assessment had a moderate positive association (r=.33) with 
the use of performance assessment and was found to explain 1 1 .2% of the variation in instructors 
use of the assessment method (Table 13). Attitude toward assessment was also found to explain 
4.3% of the variation in instructors’ use of portfolios as an assessment method (Table 1 1). 




168 

154 



Table 13 



Semi-Partial Regression of Use of Performance Assessment On All Independent Variables 
(n = 158)(Stepwise Entry) 



Variable 


sR' 


b 


t 


D 


Attitude 


.112 


.248 


4.33 


<.001 


(Constant) 


2.45 









Standard error = .742 
R^=.112 

Adjusted R^= .106 

For model: F = 18.73, p <.001 



Potential constraints faced by instructors during the assessment process explained 2.8% 
of the variation in the instructor’s use of standardized test scores while the program area of 
Health Occupations explained 9.2% of the variation (Table 14). There was not a statistically 
significant association between attitude toward assessment and use of portfolios, yet when it was 
entered into a regression model with three other variables (marketing, constraints, competence) it 
explained 4.3% of the variation in instructors’ use of portfolios as an assessinent method (Table 
11 ). 
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Table 14 



Semi-Partial Regression of Use of Standardized Test Scores On All Independent Variables 
(n = 158)(Stepwise Entry) 



Variables 


sR^ 


b 


t 


P 


Health Occupations 
(Program Area)* 


.092 


.405 


3.14 


<.001 


Potential Constraints 


.028 


.555 


2.19 


.03 


(Constant) 


.329 









Standard error =1.01 
R^=.120 

Adjusted =. 1 08 

For model: F= 10.11,2 <.001 

“Multichotomous variables were dummy coded: Health = 1 and Other = 0 
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Competency level was found to contribute only 3.3% of the variation in instructors’ use 
of portfolios (Table 11). While competence accounted for a small percent of variation in 
instructors’ use of portfolios as an assessment method, it did not explain a significant amount of 
variance in the use of objective paper and pencil methods, standardized test scores, performance 
assessments, informal observations, and essay assessment methods. 

CONCLUSIONS AND RECOMMENDATIONS 

This descriptive study was designed to describe instructors’ use of assessment methods in 
post-secondary technical and occupational classrooms and laboratories. The study further 
investigated the relationships between use of student assessment information and characteristics 
of post-secondary technical and occupational instructors in the State of Ohio. 

Conclusions 



This section contains the conclusions reached based upon the findings from this study. 
As a reminder, an attempt was made to control for non-response error by comparing non- 
respondents to respondent. Over sampling of the non-respondents was necessary to obtain a 
sufficient number of respondents to make a comparison using a t-test. Therefore, the results of 
this study cannot be generalized to the population. 

Use of Assessment Information 

Respondents reported using objective paper and pencil methods the most for student 
assessments. Since technical and occupational instructors teach subjects that involve many 
student performance activities, it was surprising that objective paper and pencil methods of 
assessment were used more than performance assessments. In previous research, Kershaw 
(1993) found that secondary vocational teachers tended to place more emphasis upon the use of 
performance assessments than on other assessment methods. 

Standardized test scores were found of little use in making instructional decisions by the 
respondents. This tends to indicates that instructors use of standardized testing continues to be 
little by instructors in making educational decisions. 

Portfolios were found the least used source of student assessment information for 
decision-making. Although this assessment method continues to be promoted as an effective 
assessment method within education, instructors who responded tended not to use the 
information generated from portfolios. 

Potential Constraints to Assessment 

Post-secondary technical and occupational instructors within this study were split 
between agreeing and neither agreeing nor disagreeing regarding the potential constraints to the 
assessment process. The results of this study were similar to those findings reported by 
Gullickson (1984). Kershaw (1993) also found similar results in his study of secondary 



vocational teachers. In addition to the predominately neutral findings, other studies found that 
limited time for planning was the constraint that received consistent higher levels of agreement. 

Instructors’ Attitude Toward Assessment 

For the most part, the respondents had positive attitudes toward assessment and agreed 
with those reported by Green (1990) where attitudes of both preservice and experienced teachers 
toward classroom assessment were positive. Additionally, Kershaw (1993) found that secondary 
teachers tended to have a positive attitude toward assessment. 

Perceived Level of Competence in Assessment 

Technical and occupational educators in this study reported that they perceived 
themselves to be moderately to extremely competent in the assessment process. These results 
were in line with findings from Gullickson and Hopkins (1987) that described teachers as being 
comfortable in their knowledge and abilities related to the assessment process. Dorr-Bremme 
(1983) also concluded that teachers perceived their use of assessment techniques as accurately 
measuring the effects of their instruction. 

Results from this study, in the area of the competency level of instructors in assessment, 
conflicted with other research. Previous research found that teachers were lacking the necessary 
skills in assessment selection, development, and use (Newman & Stallings, 1982; Carter, 1984; 
Hills, 1991). In addition, Gullickson and Hopkins (1987) and Schafer and Lissitz (1987) 
supported the position that preservice courses have not been adequate in developing the level of 
desired assessment skills in teachers. 

From previous research, there has been evidence to suggest that teachers may tend to 
exaggerate their level of competence in assessment when data is obtained in a self-reported 
manner. Marso and Pigge (1989) found that teachers, principals, and supervisors do not agree on 
the proficiency level of teachers’ test construction and test planning proficiencies. Teachers 
tended to rate themselves as much more proficient in assessment than did their supervisors or 
principals. Findings reported by Green and Williams (1989) indicated that teachers with less 
training in tests and measurements perceived themselves to be more knowledgeable about 
interpreting standardized test scores than teachers with more training in this area. If the 
instructors’ perceived competence levels can be interpreted as being exaggerated, then the 
findings that 42.4% of instructors reported a moderate level of competence in assessment 
becomes more important. The higher levels of perceived competence, the small significant 
relationship between perceived competence and the use of portfolios, and the past research that 
identified the deficiencies of teacher assessment skills suggest the potential need for the 
upgrading of technical and occupational education instructor competence in assessment practices. 

Relationships Between Dependent and Independent Variables 

Program areas had a substantial association with the use of essays as an assessment 
method. This finding tended to indicate that respondents from marketing and business 
occupations used essay methods more than their counterparts in other technical and occupational 
programs. Competence in assessment had a low positive association with instructor use of 




information from portfolios as an assessment method indicating that respondents who perceived 
themselves as more competent in assessment tended to be more likely to use portfolios for 
assessment. 

A moderate relationship (r=.33) was present between attitude and use of performance 
assessment, which conflicts with some previous research. Green and Stager (1986) concluded 
that the relationship between attitude and assessment use was of no practical importance. 
Kershaw (1993) also concluded that the relationship between attitude and assessment use was of 
little importance. The results of this study indicated that although attitude may not be important 
in the use of all the different types of assessment methods, it was of some importance when using 
performance assessment. 

Regression of Dependent Variable Upon the Independent Variables 

This study failed to demonstrate that instructor characteristics accounted for much of the 
variation in their use of assessment information. Age, sex, years of teaching experience, years of 
related work experience, and educational level did not account for a significant proportion of the 
variation in the respondents’ use of any assessment method. 

Though there was not a statistically significant association between program area and 
standardized test scores, it was found that the health occupations program area explained nine 
percent of the variation in the instructors’ use of standardized test scores. This may have been 
due to the use of standardized test scores as an entry and exit requirement for students in many 
health occupation programs. 

Program area was found to have a substantial association with the use of essay methods 
for making educational decisions. Two program areas (marketing and business occupations) each 
accounted for three percent of the variation in the use of essay methods in making instructional 
decisions. This indicated that essay methods may be used more by instructors in marketing and 
business occupation programs than instructors in other program areas. 

This study was unable to support the claim that potential constraints faced by instructors 
while conducting assessment activities played a major part in explaining variation in instructor 
use of assessment methods. The use of two assessment methods (portfolios and performance 
assessments) were shown to be related to constraints faced by instructors while conducting 
assessment activities, but a very small proportion of variance was explained by these constraints. 

Competence accounted for a small percent of variation in instructors’ use of portfolios as 
an assessment method, however, it did not explain a significant amount of variance in the use of 
objective paper and pencil methods, standardized test scores, performance assessments, informal 
observations, and essay assessment methods. These findings were slightly different from what 
Kershaw (1993) found. Kershaw found that competence in assessment contributed a significant, 
yet small portion of variance in teachers’ use of objective paper and pencil methods, use of 
performance assessment and use of informal observation. 
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Recommendations 



Recommendations from the results of this study will be made for practice and future 

research. Based upon the review of literature, the findings of this study, and the previous 

conclusions, the following recommendations are proposed. 

Recommendations for Research 

1 . Because this study was able to explain only a small amount of variance in post-secondary 
technical and occupational instructors’ use of assessment methods, it is recommended 
that each of these methods be studied independently. Also, it is suggested that qualitative 
research be conducted on each of the assessment methods to help identify other variables 
that may contribute additional variance in explaining the use of assessment information in 
making educational decisions. Open-ended responses could be collected regarding 
instructor attitude toward assessment, competence in assessment, and potential 
constraints in the assessment process. 

2. Since attitude toward the overall process contributed little to understanding the use of 
assessment by instructors, it is recommended that fixture research relate this attitude 
toward the specific assessment methods. What are instructors’ attitudes toward objective 
paper and pencil assessment methods, standardized testing, performance assessment, 
informal observations, portfolios, and essay methods? Information gathered on the 
attitudes toward each assessment method may explain more about the use of these 
methods by educators. 

3. Individual competence in assessment was rated rather high by instructors in this study, 
yet it only explained a small proportion of variation in the use of assessment methods. It 
is recommended that firture research investigate the level of competence of the instructors 
in each assessment method. Information generated on the level of competence with each 
assessment method may find different levels of competence within each assessment 
method. Research such as this may also identify strengths and weaknesses that 
instructors have related to the assessment process. This data would be valuable to in- 
service and pre-service teacher education programs in updating their courses and 
workshops. 

4. Instructors rated their level of competence on the high side in this study, yet it explained 
little of the variation in the use of assessment methods. This may indicate that instructors 
overstated their competence level. For future research on the level of instructors’ 
competence, it is suggested that self-reported competence levels be compared with 
observed instructors’ competence level. In addition, since competence in certain areas has 
been found to be related to education within that area, it is recommended that research be 
imdertaken to collect information at the different colleges and universities to learn how 
many of the courses in the pre-service technical and occupational education curriculum 
concentrate on assessment methods and what assessment methods are emphasized within 
these courses. 



5. To provide technical and occupational instructors with ideas on how the different types of 
assessment can be used within their specific programs, it is recommended that research be 
conducted to investigate and collect examples of how instructors (within the different 
program areas) use the different assessment methods. If instructors can see specific ways 
that these assessment methods are used, they may be better able to carry out these 
assessment techniques within their individual programs. 

6. Recent emphasis on the development of national skill standards will have an impact upon 
assessment in technical and occupational programs. Future research should investigate 
how and whether these standards are being used as an assessment tool. 

7. It is recommended that the findings from this research be compared with what is being 
taught in pre-service and in-service teacher education programs. It may be that 
instructors are only assessing students with methods with which they are familiar. 

8. The general population has started calling for public educational institutions to be more 
accountable. One measure of accountability is the level of proficiency of the students. 
Students tested with a variety of assessment methods will provide more evidence to the 
public that these students are learning what they are supposed to learn. Consequently, 
instructors need to be able to use different kinds of assessment methods. 

Recommendations for Practice 

1 . Findings from this study indicate that certain demographic characteristics have little 
bearing on the use of different assessment methods. This suggests that either assessment 
practices change very little as an instructor gains instructional and educational 
experiences or that instructor assessment strategies and preferences developed early in 
their careers are changed very little over a period of time. Therefore, it is recommended 
that pre-service and in-service educators provide a curriculum that includes different 
assessment methods and how these methods can be used in the different program areas. 
Post-secondary educational institutions should provide some means of recognizing those 
instructors that use different assessment methods within their individualized programs. 

2. Post-secondary technical and occupational instructors within this study were not using 
standardized test results. Such scores may be helpful to instructors in determining 
strengths and weakness of their students. Instructors within the different program areas 
may need further education about how they may use standardized test scores to enhance 
their programs and student achievement. 

3. It is recommended that technical and occupational instructors continue to use 
performance assessments within their programs. This assessment method continues to an 
appropriate method for measuring learning performance-based environments. 

4. Data gathered from this study indicated that the different types of assessment methods are 
not evenly used. This could be due to the lack of understanding by instructors of other 
assessment techniques. The different assessment methods may be prime candidates for 
topics of in-service opportunities for instructors. 

5. Student achievement levels can vary by the different ways they are assessed, so it is 
important students are provided the opportunity to demonstrate their knowledge and skill 



level in a variety of ways. Within the field of education, certain assessment methods tend 
to be used more than other assessment methods. Instructors should determine how and 
whether these assessment methods can be used within their program areas, but they 
should not shift all of their assessment activities to just one method. 

There is a saying that “we teach how we are taught”. If this is true about teaching, then it 
may also be appropriate for assessment. University faculty, in pre-service teacher 
education programs, should use different types of assessment methods within their 
classes to serve as examples for their students. 
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Abstract 



The purpose of this study was to determine the nature and extent of continuing education and 
management training programs used by apparel retail companies; and to document their 
programs according to their content, objectives, methods, assessment techniques, and evaluation 
procedures. Methodology was based on qualitative research, using the multiple-case study 
method, with an open-ended questionnaire. Interviews of apparel retail managers and trainers 
gave insights into: (a) the content of education and management training programs, (b) the 
training methods, (c) the assessment tools, and (d) the evaluation techniques used in those 
training programs. 

The findings show that, since the restructuring of the selected companies during 1995 and 1996, 
overall educational reimbursement to employees had decreased. Training programs used both 
on-the-job and classroom methods, and were conducted primarily in stores. Managers were 
responsible for most of the training. Assessment tools varied, with observational techniques, 
examinations, self-evaluations, and reviews reported. Participants commented on the lack of 
overall program evaluation, training consistency, and communication. 

Conclusions of the study indicate that management training content was limited to technological 
advances and had little content pertaining to the changing demographics of employees. Methods 
and assessment tools used in training were conducive to learning. Program goals and objectives 
were not always clearly defined. Evaluations of the overall effectiveness of a training program 
were rarely conducted. 

Recommendations are that retailers and educators should: (a) update programs to meet the 
technological and human relation needs of a changing retail environment, (b) consider all 
company employees as potential trainers, (c) utilize different assessment techniques and methods 
of teaching, (d) standardize and evaluate training programs, (e) improve communication among 
retail company personnel, and (f) continue to develop curricula that include communication, 
feedback and assessment techniques, and field practica. 




O 



Introduction 



The changing workplace, the reorganization of retail corporations, and the growing need for 
cooperation among industry, government, and education have made many retail companies 
reassess the need for the education and training of their workforce. Retail companies have 
merged and acquired other retailers. Since the late 1980s, the industry has changed from a 
seller's market to a consumer-oriented one. Government regulations pertaining to employees and 
consumers have been increasing. All of these have made the assessment and evaluation of 
education and training opportunities an important function of retail organizations. As Crawford 
and Webley noted, "Continuing education and training of the workforce is rapidly emerging as a 
crucial problem for the 1990s in the USA, the UK, Canada, and indeed among all developed 
nations" (1992, p. 1). 

Education and development programs are needed by both apparel retail managers and their 
orgamzations. The evolution of the company, operational changes, smooth succession of 
managers for top-level positions, job performance and improvement for managers, and 
preparations for advancement are equally important to both the careers of managers and the 
survival and growth of businesses (Black, 1979). The education and development of new and 
experienced retail managers is fundamental for the growth of a retail company. 

Although their goals are different, there is a need for collaboration among businesses, 
government, and higher education facilities to expand the ability of managers to educate 
themselves and the workforce (Matthews and Noorgaard, 1984). This study is important both to 
educators and retailers. “The emphasis on educational accountability requires that students 
pursuing retailing careers be prepared through related coursework” (Woodward, 1992). 
Furthermore, high persoimel turnover rates are an expensive problem within the retail industry. 
The cost of training personnel is high. According to Pettijohn and Pettijohn (1994), the amount 
spent on retail sales training in 1990 averaged $3,737 per trainee. The use of successful tr aini ng 
techniques could reduce the costs of educating new personnel and of retraining bumed-out 
managers, and lower turnover rates. 



Purpose and Objectives 

The major purpose of the study was to explore, document, and describe the nature and extent of 

continuing education and management training programs used by apparel retail companies. 

Questions of the study included, but were not limited to: 

1. What were the content and objectives of the continuing education and management training 
programs used by the sample retailers? Were these programs job specific and technical or 
did they also include “human relations” topics? 

2. Who did the training? Where and when was training held? 

3 . What methods and assessment techniques were used by the trainers? Did the trainee receive 
feedback? 
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4. Were evaluation procedures used in the companies' training programs? How did the trainer 
measure the effectiveness of a particular program? How were goals and objectives of the 
total program evaluated? 

An additional purpose of this study was to inform educators and retailers about retail 
management training practices in order to better prepare their students and employees. Studying 
the content and methods used in retail management programs could help educators develop 
curricula that will help their students to be knowledgeable and prepared for the workplace. 

Theoretical and Conceptual Base and Related Literature 

The education and development of new and experienced retail managers is fundamental for the 
growth of a retail company. Prior to World War II, management training programs were rare 
(Black, 1979), However, by the 1990s, many retail companies had organized human resources or 
training departments that oversaw the development of their managers' education. 

The pedagogy of retail management training has developed into a science similar to teaching in 
an educational institution. Certain elements are identical in planning a successful educational or 
training program. A trainer is a teacher who must understand both the company's and the 
trainee's philosophies, objectives, and goals in order to plan a program that is profitable for both 
the company and the employee. To develop the best methods and resources to reach those aims, 
a trainer should continually assess and evaluate instructional aids, the training program, and the 
comprehension and resulting behavior of the trainee (Callahan, Clark, and Kellough, 1992). 

"Raising Retail Standards," developed by the National Retail Federation (Reda, 1995), as well as 
Ahern's (1982) study, concentrated on training for entry-level positions in retailing. McCord 
(1983), for the National Institute of Education, documented tuition reimbursement by retailers. 
Training theory research identified various categories of variables that enhanced learning 
effectiveness. Three of these— learning experience, individual characteristics, and work 
environment— were recognized by Baldwin and Ford (1988). However, Berardinelli, Burrow, 
and Jones (1995) reported that practical impact on organizational performance is what makes 
training significant. The practical variables Shepherd and Ridnour (1995) examined included 
variables in methodology of training. Although the Shepherd and Ridnour study was limited to 
sales training, not retail management training, the analysis of methods and contents laid a 
foundation for this study. 



Research Methods and Procedures 



Sample 



The sampling design was goal-oriented, as defined by Wiersma (1991). The goal was to 
interview personnel in apparel retail stores’ home offices or training departments. Wiersma 
(1991) also stated that practicality and economy are justification for selecting a particular sample. 
Thus, the sample selected was located in one particular state. A list of retail companies was 
randomly selected from a list of businesses meeting the following criteria: 
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1 . Operated a minimum of 50 nationally located branches, as listed in Fairchild’s Retail 
Stores Financial Directory 1996 (Benj ami n, 1996). 

2. Listed in the state’s “Service Directory, 1994-1995,” classifying businesses by the 
U.S. Office of Management and Budget’s Standard Industrial Classification fSICf 
Manual n987L 

3. Represented (SIC) industry groupings — 53 1 1 , department stores; 5611, men’s and 
boys’ clothing and accessories; 5621, women’s clothing stores; 5632, women’s 
accessory and specialty stores; children’s and infants’ wear stores; 5651, family 
clothing stores; 5661, shoe stores; or 5699, miscellaneous. 

The interviewees within each company were selected for their expertise and their different 
perspectives about apparel retail management training and education. Two trainers and three 
managers from each of three companies were interviewed. The selection of the interviewees 
reflected replication logic and developed descriptive scenarios. This balance in the choice of 
interviewees represented different perspectives within the study as Rubin and Rubin (1995, p. 
69) suggested. 

Since the purpose of this qualitative study was to explore, document, and describe the arena of 
apparel retail management education and training, the use of this sample documented both the 
variations and the similarities of those programs. This purposeful sampling (Seidman, 1991)-- 
defming terms, determining the range of population, narrowing the population by sites, and 
selecting the levels of groups within the sites-enabled the manageability of that population. 
Descriptive and conceptual categories were able to be developed. 

Instrument 



A fourteen page questionnaire was developed for the interviews. Questions were developed after 
completion of a literature review of studies concerning qualitative research methods and an 
overview of the apparel retail industry, including management turnover, education, and training 
methods. Questions were written to document apparel retail education and training— its content, 
assessments, and evaluation. A panel of experts including a researcher, a trainer, and a manager 
(who were not involved in this study) reviewed the questionnaire. Closed-ended questions were 
used for demographics including length of employment, position, and experience. Open-ended 
questions were employed to group similar topics such as: pedagogical ideas, educational 
objectives, methods of training, assessment, evaluation, and perceived results. 

Procedure 



The use of qualitative research and the multiple-case study with an interview technique provided 
an appropriate method for examining the content and objectives of the education of retail 
managers through management training programs in apparel retail organizations. I.E. Seidman’s 
(1991) method of conducting interviews was adapted for this study. The study's design helped to 
maintain data collection protocol and allowed for the use of grounded-theory analysis. 
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The study’s boundaries were limited to three apparel retail companies in a selected state. The 
three store types selected were a department store, a specialty store, and an off-price apparel 
store. To ensure validity of data, a detailed plan for collected data was established. First, 
contact was made by telephone. A request for interview, confirmation letter, and follow-up prior 
to the interview were repeated in each case. After the interviewee signed the consent form, the 
interview was audio-taped and the interviewer administered the same questionnaire with each 
participant. 

A multiple-case study format was developed. Replication logic in this multiple-case study was 
analogous to a quantitative study’s use of multiple experiments in sampling logic (Yin, 1994, p. 
45). That is, interviews, data categorizing, and analysis for each of the three companies that were 
examined were conducted in the same way. Field notes, other notes, and transcripts of the 
interviews were made. This would enable replication for any future study. 

Procedures for data management, reduction, and display were developed. Data were transcribed, 
categorized, indexed, and examined for analysis using the Non-numerical Unstructured Data 
Indexing Searching and Theorizing computer package (QSR NUD-IST®) (Qualitative Solutions 
& Research Pty. Ltd., 1996). Steps in data analysis were identified. Each interview was 
recorded and transcribed. The interviews were stored into the data analysis computer program, 
using text only data as an on-line document. The text was reviewed and classified by subject 
areas and classifications, and segmented by units. Each text unit was examined and defined. 
Indexing and patterning of classified units enabled the researcher to analyze the documents by 
text unit category and to interpret the data within a thematic framework. 

The study’s design incorporated a three-in-one multiple-case design. Data were gathered from 
three different retailers and analyzed as one unit. The entire analysis and report of findings 
consisted of cross-case analysis with individual cases dispersed throughout the findings. This 
method of analysis, multiple sources, and protocol helped to maintain anonymity of the subjects 
and increased the reliability of the study. 

The strategy used for analyzing this research was based on the grounded-theory approach. 
Categories were generated until theoretical saturation was met. That is until no other new 
categories or themes were found. Grounded-theory analysis not only permitted the categorizing 
of data, but also allowed for linkages. This strategy built more reliability into the study. 

In order to use grounded-theory analysis, the researcher followed the following steps, as adapted 
from Bryman and Burgess (1994, pp. 4-7): 

1 . immersion into data by listening to the interviews and reading the transcripts, 

2. examination of the study's objectives by using the questionnaire and identifying 
themes, 

3. labeling of topics and looking at developed patterns, 

4. coding of each individual transcript, and 

5. looking at patterns, attitudes, and experiences of the interviewees. 
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Two basic principles of qualitative research were followed: First, completeness was met by 
choosing people that are knowledgeable about the subject. Interviews were conducted and the 
subjects were allowed to talk until a sense of the meaning of a concept or a process was 
understood. Second, saturation was brought about when the narratives repeated the same events 
and similar varieties of interpretation occurred (Rubin and Rubin, 1995, p. 73). 

Findings 

Data collected for the study of the education of retail managers through management trai nin g 
programs in apparel retail organizations consisted of non-numerical text units from 15 interviews 
of managers and trainers within three apparel retail companies of a selected state (Table 1). A 
list of 52 nodes (categories) used in the study was extrapolated from 14,125 text units. 
Information concerning the objectives and questions of the study was documented and analyzed, 
using either descriptive or conceptual categories. 

Table 1 

Positions and Experience of Interviewees 



Position 


n 


Average Years in Retail 


Director of Training 


3 


11 


Training and Development Manager 


3 


8 


Sales Manager 


3 


6 


Store Manager 


6 


10 


Totals 


15 


9 



The following are examples of the interviewees’ perceptions to the study’s questions. 

1. What were the goals, content, and objectives of the continuing education and management 
training programs used by the sample retailers ? Were these programs job specific and technical 
or did they also include “human relations” tonics ? 

Restructuring of companies, both consolidation and expansion, was affecting the organizational 
structure of these retailers. Maintaining profitability and the continuity of management 
personnel within these companies were the most important goals of training. (Other goals both 
company oriented and personnel oriented are reported in Table 2.) One person concluded; 

It’s the bottom line. It’s how we’re going to increase sales and, you know, the two 
major goals of our company are to increase sales and develop people. They both 
tie in, if you develop people you will increase sales and those are the two priorities. 

And, to do that, training is obviously a major initiative in developing people. 

The managers that were interviewed said that management and staff shortage was a major 
problem and that to train a potential manager was time intensive. For example, one manager, 
concerned about staffing problems, said: 
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Well, they plan on opening more stores. However, they’re consolidating some of 
the management. Some of the smaller stores are going to have one manager for 
two stores as opposed to two. So, it’s going to affect everybody. 

The interviewees pointed out that many managers are career-oriented and that they wanted to 
improve their skills. “Some people, who certainly want to get ahead, usually end up taking 
material home and learn themselves.” One trainer thought about learning and training this way: 

I think that’s human nature— most people want to learn and want to grow in a 
position. I would imagine that would be a motivating factor. I think a lot of 
people equate personal development with also getting ahead. So, if I want to get 
to the next level; what type of things do I need to work on for myself, right now? 



Table 2 

Goals in Management Training Programs as Reported by Interviewees 



Company Goals 


Management Trainee Goals 


Build a Solid Team 


Career Movement 


Consistency 


Wanting to Learn 


Structure 


Growing in Your Position 


Strengthen the Company 


Going Forward 


Internal Promotion 




Increase Productivity 




Continuity of Managers 





The training content in these programs was mainly job specific. “Teaching people different 
procedures in the store,” was one manager’s answer. The attainment of these objectives included 
operating procedures and fashion theory — “We call it merchandising.” However, home office 
trainers reported that the soft-skill, human relations areas , such as time management, 
communication, leadership, and others, were also included in training competencies. 

2. Who did the training ? Where and when was training held ? 

Managers conducted most all of the in-store training, usually technical and store operation skills. 
As has previously been mentioned, training managers is time consuming. The time constraints 
of store or department managers left teaching the trainee lessons “on the side” and in between 
times. Human characteristics play an important part in how and what each manager taught the 
new managers. One manager stated that attitude was everything: 

On my level I try, I try to make it— to have my employees be more responsible for 
themselves. When I interview someone I stress right up front that they have to get 
to work on time. I was stressed that at a young age. I grew up on a farm and I 
always had stuff to do. But, sometimes you don’t want to go to work, but in 
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reality most people have to. I stressed get here on time. I think that’s the most 
important— it starts your day off— it’s, let’s get up and know you have to do this 
and not, you know, say you didn’t have a job, I mean your attitude towards 
everything is just down the tubes, I think. 

Some training, usually the soft-skill training sessions, was conducted in corporate home offices, 
conducted by home office or human resources personnel. All the interviewees said that tr ainin g 
was held during the trainee’s working hours. Training sites were most often located within the 
store, with some off-site training being conducted. One manager commented. 

Through experience, it’s the best way and really the only way. You can be taught 
in the classroom, but you actually are thrown into a situation — ^presentation, 
where it’s real and it’s live and you have to make a decision. 

3. What methods and assessment techniques were used by the trainers ? Did the trainee receive 
feedback ? 

The methods used most often for training were hands-on, on-the-job training for technical 
objectives. Classroom, role play, and other methods were used to teach human relation 
objectives (Table 3). Mentoring was mentioned by both managers and trainers. Geiger-DuMond 
and Boyle (1995) suggested, a well-designed mentoring program will help develop strong 
managers. The participants of this study acknowledged that mentoring is a growing method in 
retail training. One interviewee said: 

And I was very fortunate to have another department manager who was my 
buddy, who I worked really close with. Because, when we first opened the store, 
they over hired managers. Then it decreased, slowly. So, I was working very 
closely with her and she took me under her wing and trained me. So, I was very 
fortunate to have that happen to me. 
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Table 3 

Methods Used In Retail Management Training Programs of Study Participants 



Methods 
On-the-Job 
Classroom Lecture 
Seminar 
Coaching 
Delegation 
Mentoring 
Buddy System 
Role Playing 
Discussions 
Case Studies 
Videos 

Training Manuals 
Checklists 
Group Training 



When asked what assessment tools were used in their management training programs, the 
interviewees responded that observation was used most often. Other assessment tools, such as 
exams, were also used. “I’m a firm believer in ‘inspect what you expect.’ If you get 86% on the 
skills assessment, congratulations we’re going on.” 

The interviewees said that feedback to the trainee was usually immediate. Self-evaluation 
techniques and written performance evaluations were used. Managers used self-assessment when 
quarterly reviews, promotions, or pay raises were discussed. Self-evaluation was also used in 
conjunction with action plans. In one case, a store manager was training an assistant manager to 
be a store manager. She spoke about how she used self-evaluation techniques: 

Yes, we would ask them to [do them] more formally as the assistant grows with 
the company, self-evaluations are built into every evaluation period. So, for any 
position in the field they should be doing a self-evaluation ... the store manager 
would ask an assistant, ‘where do you rate yourself?’ 
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Table 4 

Assessment Tools in Retail Management Training Programs as Reported by Interviewees 



Assessment Tools 
Observation 
Oral Questioning 
Group Presentations 
Checklists 
Action Plans 
Written Tests 
Self-Evaluation 
Performance Evaluations 
Secret Shops 



4. Were evaluation procedures used in the companies' training programs ? How did the trainer 
measure the effectiveness of a particular program ? How were goals and objectives of the 
total program evaluated ? 

Determining training program evaluation techniques varied within these companies. From the 
store level perspective, some interviewees considered quarterly reviews and checklists as means 
of evaluating program effectiveness. Evaluation procedures included, but were not limited to, 
performance reviews, written examinations, and observations (see Table 5). Precise program 
evaluation was not performed regularly, if at all. A trainer commented; 

I honestly would like to see more of that [total program evaluation] done. [The 
training manager] and I talk about [it], but we lose that Teaming evaluation.’ But, 
we’re working on ways now to gauge how has their performance changed after it 
[the training program]. It just so hard to manage with the number of stores that 
we have. But, that’s our— one of our goals. But, it isn’t formally in place yet. We 
get a lot of informal, subjective feedback. But nothing formal. ^ 



Table 5 

Training Program Evaluation Techniques as Reported by Interviewees 



Evaluation Techniques 
Quarterly Reviews 
Observation 
Sales Per Transaction 
Written Exam 
Skills Assessment 
Class Enrollment 
Management Turnover 
Secret Shops 
Feedback 
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Exact figures for management turnover rate, which could allow for a clearer measurement of 
program effectiveness were neither kept nor known by the interviewees. Although for her store, 
one manager noted: 

[We lose] managers who have been here [as little as] two months just as quickly 
as those people that have been here [as long as] 20 years. If we lose one more 
manager it will be 100% turnover rate in a year and a half 

Communication among managers and senior personnel was perceived to be lacking by most of 
the interviewees. Disillusionment with the training program and lack of overall assessment was 
expressed. One interviewee noted; 

I think it is that we don’t deliver what we promise to deliver in an interview. I 
think we say, you know, we’ll train you, you’ll receive feedback. This is the 
expectation of the company. And, I think, a lot of people become disillusioned- 
people that have left the company. I’ve had assistants that have left, who said this 
just wasn’t what I thought it was going to be. Or, as a store manager, that did not 
do a good enough job training them initially. Obviously, there are some personal 
reasons and things like that. But, most often, associate ranks as well, we don’t 
deliver what we promised. 



ERIC 



175 



189 



Conclusions and Recommendations 



Conclusions 



This study presented the managers' and trainers' descriptions and perceptions of their companies 
retail management programs. An internal understanding of the content, objectives, methods, 
assessment, and evaluation tools used in retail management training programs can be 
appreciated. Reviewing these programs from the perspectives of the people who directly affect 
and are directly influenced by the training is an approach not used uniformly by training program 
developers, as evidenced by the lack of program evaluations described by the participants in the 
study. 

Apparel retailing in the United States has gone through many changes, from the small owner-run 
businesses of the 1800s and early 1900s, to the multi-chain organizations of today. In the 1990s, 
retail corporations, for the most part, are large conglomerates. Mergers, acquisitions, and 
restructuring of departments, divisions, and even whole companies have set the current direction 
in retail corporations. 

Most of these multi-chained retailers had recognized the need to educate and train their 
employees if their goals to satisfy the consumer and make a profit were to be met. However, 
educating and developing top retail managers comes with a cost; an outlay of both management’s 
time and company's money. According to the interviewees, neither of these resources was used 
adequately by any of the companies. 

Retailing has been in constant transition. As documented by the study, training programs also 
have been continually changing. Not only has organizational restructuring necessitated change 
in the merchandising content of training programs, but education in technological advances also 
has been included for the company to remain competitive. New technology, computers, and e- 
mail, for example, have changed the way report filing and scheduling has been done. In some 
companies, this has resulted in a need for computer classes pertaining to the companies' software 
applications. 

The consumer market also has changed. In the 1990s, the retailing industry has become more 
consumer driven than in the past. Customer service has been deterrriined to be a primary strategy 
of retail companies. Training departments, in off-price as well as specialty and department 
stores, have recognized this and have developed programs to improve the quality of customer 
service. Everyone of the participants interviewed mentioned quality customer service as being 
vital for their company. 

Not only have the consumer markets changed, but retail employees and managers face different 
challenges. Lifestyles and demographics of eniiployees and the responsibilities of managers have 
stimulated training departments to try to accommodate schedules and include more educational 
objectives pertaining to human relations. Courses in leadership training, learning styles, 
diversity, and time management have been offered. 




176 



190 



Some training programs have been designed to be both learner and company goal oriented. 
Methods used in the industry have tried to meet both of those issues by using various techniques. 
For job specific skills, such as store procedures and merchandising, manager-trainers used on- 
the-job approaches such as mentoring, buddy system, coaching, and delegation. On the one 
hand, on-the-job training gives a manager an extra person on the staff to rely on when customer 
traffic increases; but, on the other hand, finding time for a manager-trainee is difficult for many 
managers. 

Training topics such as communications, self-esteem building, and leadership have been taught 
using role playing, case studies, and group seminars. Usually, as noted in the study’s interviews, 
most of this type of training was done off-site, customarily in a training center, and with a 
company trainer. 

Although, in the study’s literature review, computer methods of training were mentioned, they 
were not suggested by interviewees within the study. The exception was the use of computer 
training for new software applications having to do with the cash register and employee 
scheduling. More hands-on and trainer-trainee methods were used in these areas by participants 
of this research. 

Trainers, managers, and literature have maintained that the day-to-day operations of a store were 
best learned on-the-job. As documented in the study, learning by doing and the guidance of a 
mentor were used to develop a quality manager able to deal with technical problems as well as 
employee relations problems. 

As some studies had indicated (Baldwin & Ford, 1988; Edwards, 1950; Bluestone, et al. 1981; 
and Berardinelli et al., 1995), training programs should also include education in communicating, 
organizing, and leadership competencies. Managers needed and wanted these human relations 
courses, but also wanted practical applications and examples so that knowledge could be applied 
in the store. 

The communication of existing and new programs was not constant in the retail companies 
studied by this research. The rapid expansion and restructuring of retail corporations had 
developed communications and consistency problems. Some personnel in the field did not know 
what programs were available to them, and if they did, they often felt that their company’s senior 
management did not listen or were not aware of store management problems. 

The time constraints that managers had to contend with made it very difficult for training new 
managers. Retail management responsibility had grown to not only include merchant-related 
duties but also included training obligations. All trainers and managers thought that a 
formalized, consistent manager training program needed the necessary time to be able to develop 
a quality manager. 

Lastly, although assessment techniques were being used for individual program objectives, 
evaluations for the overall management training programs were inconclusive. These respondents 
seemed unable to distinguish between the assessment of trainees, who have undergone company 



training, and the evaluation of the effectiveness of the training program itself The same methods 
and measures were repeated by the respondents when queried about assessment tools and 
program evaluation methods. It is also possible that none of the companies employed a 
systematic means of establishing program effectiveness. Managers and trainers both claimed 
that program evaluations were either inconsistent or non-existent. 

Implications 



This study has provided a qualitative approach to researching education and management 
training programs in the apparel retail industry. This non-statistical approach allowed the 
researcher to describe the perceptions of those interviewed; how they thought their training 
programs were in their own words. The implications of this multiple-case study research are 
drawn ifrom the interviews conducted. The descriptions and explanations are as were perceived 
by the interviewees. Construct validity and reliability techniques were built into the research 
design, allowing for replication and induction in other cases. 

The study, in presenting the managers’ and trainers’ descriptions and perceptions of their 
companies retail management programs, provides an internal understanding of the content 
objectives, methods, assessment, and evaluation tools used in retail management training 
programs. Reviewing these programs from the perspectives of the people who directly effect and 
are directly influenced by the training is an approach not used uniformly by training program 
developers, as evidenced by the lack of program evaluations described by the participants in the 
study. 

This multiple-case study explored and described the education and training used by three major 
retail apparel companies to develop their managers. The interviews told the stories of those 
directly influenced by the programs. Experienced and new managers and trainers commented on 
the questions raised by the study. These individual observations documented a picture of what is 
the real world in the management education and training programs of these retail companies, 
today. 

Recommendations 



The development and application of a successful apparel retail management training program 
could help satisfy the primary goals of companies in the retail industry, specifically to make a 
profit and satisfy the customers. The analysis of the findings in the study has resulted in the 
following recommendations: 

1. The content of management training programs for apparel retail managers should be 
continually updated to include both new technological advances in retailing and human relations 
areas to accommodate the changes in retail technology and the changes in personnel lifestyle 
needs. Since the apparel retail industry is changing constantly, the training program directors 
should update the structure of their training programs accordingly. 
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Management trainee selection could be benefited with the development of specific criteria related 
to a particular store’s needs. Companies should continue to look at internal and external sources 
for management staffing in order to bring fresh ideas and continuity to the management process. 

2. The training programs should continue to include all store employees— supervisors and 
experienced employees, as well as managers and trainers to help train new managers. 

Specific times need to be scheduled, away from the store, to allow managers to train their 
manager candidates. Training schedules need to be flexible to oblige the different learning styles 
of trainees as well as the store’s traffic flows and business needs. 

Training departments should continue to use all types of learning methods. In particular, 
mentoring with an experienced manager should be encouraged. A structured program should be 
developed. 

Off-site training locations, away from the store, are needed for managers to attend seminars. 
These should be scheduled at regular intervals, more than once a year, but not every week. This 
time allows managers to compare notes and to discuss common problems, in addition to attaining 
the seminar’s intended objective. 

3. The trainers should continue to assess their trainee’s progress in meeting specific educational 
objectives. The trainer-managers and company trainers should continue the use of observational 
techniques, oral questioning, written exams, and project completion. Prompt and immediate 
feedback should be provided to trainees regarding their progress. 

4. The company program directors should evaluate their apparel retail management training 
programs. Techniques for evaluating the overall effectiveness of training programs should be 
developed and implemented. Program goals and objectives must be clearly defined and 
communicated to trainers and trainees. 

All managers who leave an organization should complete an exit interview. An analysis of 
managers’ reasons for leaving the company could help with developing or changing specific 
training programs. Manager turnover rate should be available to the training program directors. 

5. The companies should address the need to improve communication among entry-level, 
middle, and senior management. It is not what is happening that is most important, it is knowing 
what is happening. Company loyalty and employee satisfaction increases with both informal 
(oral) and formal (written) communications. 

The combination of time and money, communication, consistency, evaluation, and 
standardization in training is paramount if a company wants to satisfy its customers and continue 
to be profitable. 

6. An equally important recommendation is that educators in post-secondary institutions 
continue to emphasize courses in communications and feedback and assessment techniques. 



Field practica and internships should be used to supplement classes and provide students with 
industry-based experiences. 

For Further Study 

The significance of the study was founded in the relevance of its practical applications. The 
issues that were discussed were of importance to the retail industry and to educators of retail and 
merchandising students. In retail, senior and middle management need to communicate with 
their managers and trainers. In academia, educators need to communicate with retailers. In both 
instances, the communication of needs and wants could help to improve education, promote the 
growth of companies, satisfy consumers, and increase job performance and satisfaction. 

More studies documenting other retail management training programs are essential. Replication 
of the study, using the same methodology, very possibly could result in approximately the same 
results, because of the researcher’s use of the multiple-case study format. However, the use of 
different companies, different trainers, different managers, at a different time in a different state, 
would enhance even further the reliability of the study. 

In conclusion, this qualitative study gave an insight that quantitative research could not have 
realized. People— what they thought, how they perceived situations, and why they did what they 
did, giving their own interpretations of what their company’s retail management training 
program was— have brought forth an awareness and description of the retail training program 
arena. They told what their experience was. 

Since retailing is people intensive, attaining and retaining quality people is necessary for the 
survival of apparel retail companies. The development and implementation of a quality 
management training program could help in meeting the goals of retailers. 

The development and application of a successful apparel retail management training program 
could help satisfy the primary goals of companies in the retail industry, specifically to make a 
profit and satisfy the customers. Successful rnanagement training programs could also help 
satisfy employee needs of self-fulfillment and job performance. 
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ABSTRACT 

This study was designed to document the status of work-based learning programs 
(cooperative education and youth apprenticeship) in Georgia secondary schools. The 
perspectives of work-based learning coordinators, students, and employers were sought 
in order to provide information about how these programs are implemented and, more 
importantly, how to improve these programs. To address the purpose of this study, a 
survey research design was utilized. Data were collected from the three sub-populations 
with separate questionnaires. On the basis of data from each of the three sub- 
populations taken as a whole, nine themes were derived related to structure of the work- 
based learning programs and five processes themes by which they are implemented. 

Introduction 

Work-based learning through school-to-work programs, which contributes to both 
the intellectual and career development of high school students, is gaining acceptance by 
policymakers and educators as a means to improve the educational outcomes for many 
students. This increased acceptance of work-based learning coordinated by the local 
school has been brought about by changes in the U.S. and world economies, the 
American educational system, and the increased focus on the importance of preparing all 
students for the world of work. 

As defined by the School-to-Work Opportunities Act (1994), a school-to-work 
program must include a school-based learning component, a work-based learning 
component, and a connecting activity component. The work-based learning component 
must include: (a) work experience opportunities for students; (b) job training and work 
experiences coordinated with learning in school-based programs that are relevant to 
students’ career major choices, and lead to the award of skill certificates; (c) workplace 
mentoring; (d) instruction and activities in general work place competencies, including 
positive work attitudes, employability, and practical skills; and (30 broad instruction, to 
the extent practicable, in all aspects of the industry (The National School-to-Work Office, 
1997). 



School-to-work programs that incorporate work-based learning come in many 
forms, but have the common goal of providing students vvdth experience in the world of 
work. The most common school-to-work programs available to facilitate the preparation 
of youth for transition to work include cooperative education, youth apprenticeship, 
registered apprenticeship, school-to-registered apprenticeship, clinical experiences. 
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internship or practicum, and job shadowing (Smith and Rojewski, 1993). These school- 
to-work programs arrange hands-on workplace experience for students that will provide 
them with opportunities to learn work-related skills and attitudes they could not otherwise 
acquire in a classroom. In addition, these programs may increase the students’ prospects 
for future gainful employment (Stem, Hopkins, Stone, and McMillion, 1990). 

The two most common school-to-work programs that utilize work-based learning 
in the secondary schools in the state of Georgia are cooperative education and youth 
apprenticeship. Each is a structured educational program that integrates classroom 
learning (school-based) with productive, structured work-experiences (work-based) 
which should be related to a student’s career goal. Both cooperative education and youth 
apprenticeship programs are dependent upon local business to provide work-experience 
for students and the abilities and skills of the work-based learning coordinator who has 
the responsibility for managing the program. 

A key element of both of these work-based learning programs is that they combine 
school and work during the same period of time. This may mean some hours of work 
each day, some days of work during the week, or some weeks working during the year. 
Combining school and work serves two purposes. First, it helps young people learn skills 
and knowledge to qualify for a full-time Job in the near future. Second, it gives them the 
experience of using work to foster their own learning and thus contributes to their 
capacity for change and continued growth in the longer run. 

Cooperative Education 

Cooperative education is the most commonly available form of work-based 
learning in the public schools that is designed to assist students in making the transition 
from school to work. Cooperative education is a program of instruction that features 
agreements between schools and employers to provide on-the-job experiences that relate 
to a student’s career focus. These experiences are based on objectives jointly developed 
by school personnel and an employer within the student’s career area. 

In a typical cooperative education program, employers provide part-time jobs in 
the student’s career field. A teacher-coordinator of the cooperative education program 
arranges placements, develops a training plan with the employer specifying what the 
student is expected to learn on-the-job, and makes periodic supervision visits to the 
employer’s business. The employer evaluates the student’s job performance in 
consultation with the teacher-coordinator. 

Cooperative education programs provide paid work experiences linked to the 
occupational programs students are pursuing. Students work on-the-job part-time 
(usually in the afternoon) and attend both academic and occupational-related course(s) 
during the remainder of the school day. 

There are two types of cooperative education programs in the state of Georgia. In 
the first type, the school has a separate program for a specific career area; agricultural 
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education, business and office education, family and consumer science education, 
marketing education, trade and industrial education. A school may not have a 
cooperative education program in each career area, but the programs that it does have are 
specific to instruction within that career. The second type is the combination or 
interrelated approach. In schools where student enrollment is limited or employment 
opportunities in the community are scarce a single cooperative education program may 
exist. A single program, such as diversified cooperative training (DCT), would provide 
work-based learning experiences for students in a variety of career occupations. 

Youth Apprenticeship 

Youth apprenticeship is the newest form of a work-based learning program. This 
program offers students both school-based and work-based education experiences. It 
combines structured, paid work and training on-the-job with related classroom 
instruction. The emphasis is on contextual, real world learning through work-place 
experience. 

Effective youth apprenticeship programs require extensive coordination between 
employers, schools, labor, and government because of its high academic standards and 
high-skill career opportunities. As designated by Georgia State legislation, some of the 
defining characteristics of youth apprenticeship are: 

• Employers’ active participation. Jobs, training and mentoring opportunities are 
provided to participants. In addition, employers assist in developing curricula and 
setting industry standards. 

• Integration of work-based and school-based learning. Structured classroom 
instruction and work place experiences are integrated so that one reinforces the other 
and allows for the acquisition of skills with increasing levels of difficulty and 
complexity. 

• Integration of academic and vocational learning. Cognitive and technical skill 
development, high academic standards, and infiision of each with aspects of the other 
that breaks down the traditional barriers between academic and vocational learning. 

• Secondary and post secondary linkages. Structured connections generally begin in 
the 11th or 12th grade and continue into one or two years of post-secondary 
education. 

• Award of an occupational skill certificate. In addition to the high school diploma and 
the post secondary certificate or degree, students receive a certificate of mastery of 
occupational skills. 

This combination of “real” work experience, student-mentor relationship, integration of 
education and work, and certificate of accomplishment is designed to provide students 
with structured pathways from school-to-work. 
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Purpose of the Study 



This study was designed to document the status of work-based learning programs 
(cooperative education and youth apprenticeship) in Georgia secondary schools. The 
perspectives of work-based learning coordinators, students, and employers were sought in 
order to provide information about how these programs are implemented and, more 
importantly, how to improve these programs. 

Methodology 

To address the purpose of this study, a survey research design was utilized. Data 
were collected from three sub-populations with three separate questionnaires. The first 
questionnaire “Assessment of Work-Based Learning Programs in Georgia” was 
completed by work-based learning coordinators (Smith and Thorton, 1998a). The 
population for this questionnaire was obtained from five sets of lists totaling 519 names 
of work-based learning coordinators and secondary schools from the Georgia Department 
of Education. Questionnaires were mailed to each of the identified work-based learning 
coordinators and upon completion of follow-up procedures, a total of 128 (24.7%) work- 
based coordinators responded. 

Students currently enrolled in a work-based learning program provided data on the 
second questionnaire “Student Questionnaire on School-Sponsored Employment” (Smith 
and Thorton, 1998b). To arrive at a list of students within work-based learning programs 
to be surveyed, a cluster sampling approach was used. The sampling frame for this 
questionnaire was obtained from the five sets of mailing labels of work-based learning 
coordinators and secondary schools from the Georgia Department of Education. From 
the list of 519 work-based learning coordinators, 64 were randomly selected utilizing a 
table of random numbers. Each of the work-based learning coordinators selected was 
contacted by telephone to solicit their participation and to identify the number of students 
in the work-based learning program who were engaged in an actual work-based learning 
situation. A total of 40 work-based learning programs (62.5%) returned 878 student 
questionnaires. 

The third questionnaire “School-Sponsored Employment Survey” was completed by 
employers supervising students in work-based learning programs (Smith and Thorton, 
1998c). The 128 work-based learning coordinators participating in the first phase of this 
study were asked to provide a listing of five participating employers who supervised 
work-based learning students on-the-job. The work-based learning coordinators 
identified a total of 572 employers. Each of the employer were sent the employer survey 
instrument and upon completion of follow-up procedures 325 employers (59.4%) 
provided useable responses. 

These three survey questionnaires, as well as follow-up interviews, provided a basis 
for the findings presented for this study. 



Findings 



On the basis of the data from each of the three sub-populations (work-based 
learning coordinators, students, and employers) taken as a whole, the following themes 
were derived related to structure of work-based learning programs (cooperative education 
and youth apprenticeship) and the processes by which they are implemented. 

Structural Issues 

The following structural issues for work-based learning programs were identified 
from data collected from the three sub-populations of this study. 

Advisory committee. An advisory committee for the work-based learning program 
serves as an organized base for two-way communication between the work-based 
learning coordinator and representatives from business and industry. The committee’s 
function is to advise and assist the work-based learning program on matters pertaining to 
the instructional program and policies. 

The data suggested that program advisory committees exist mainly on “paper” and 
are not actively involved with the work-based learning program. Work-based learning 
program advisory committees should be firmly maintained as they provide the impetus 
and direction for program development. 

Student identification or screening. Work-based learning programs offer students 
unique opportunities not available through “regular” in-school courses. The work-based 
learning coordinator has responsibility to help ensure that the students who can benefit 
most from the program actually become enrolled. 

Data would indicate that though there were minimum requirements for entrance into 
the work-based learning program; basically “all comers” to the program were enrolled, 
regardless of career interest or abilities. Some students may enroll in the programs only 
to be “released early” for their part-time job or “to earn money” and have little interest in 
the curriculum nor have pre-employment skills required for positions within their career 
interest. 

Job placement. Placement by the work-based learning coordinator of a work-based 
learning program involves establishing training sites (places of employment) and helping 
students select training sites that parallel their interests and abilities. Placing students in 
work-based learning situations involves preparing students for initial visits with 
prospective employers, developing training agreements and plans. 

Data denoted that there is very little job placement of students. Students often come 
to the work-based learning programs already employed in a part-time position that they 
have found on their own, or with the aid of family or friends. This position is then used 
for the work-based learning portion of the program, but may not offer actual preparation 
of the student for their selected career or stated career goal. 




Training agreement. Each student in a work-based learning site should have a 
signed training agreement that defines the responsibilities of the student, employing 
business, training supervisor (mentor), school, parent, and work-based learning 
coordinator that is signed by all participating parties and is kept on file by the school. 

Differences between stated practice by teacher-coordinators and actual involvement 
of employers were noted in the data. The training agreement seems to be viewed as a 
“compliance” item that has no real meaning in the work-based learning program. 

Training plan. A training plan is the key to effective instruction and learning at the 
work-based learning site. A training plan is a written learning plan describing the 
learning experiences the student will achieve on-the-job. The work-based learning 
coordinator and employer in consultation with the student should cooperatively develop 
the learning objectives identified for the training plan. 

Even though teacher-coordinators implied that training plans were in place for a 
majority of students, employer and student data did not substantiate this claim. It was 
deduced from the data that the employer would “train” the employee according to their 
standards for the job held by the student. Further analysis of data indicates that students 
are spending a majority of their time “doing regular work” with little opportunity to 
experience activities requiring differing skills and increasing levels of ability. 

Student evaluation. The evaluation of students’ progress in acquiring the necessary 
occupational competencies on-the-Job is a key function of the work-based learning 
coordinator and the employer. It is through evaluation of students’ progress that effective 
plans can be made to provide the students with the experience they need to achieve their 
career goals. 

Data indicates that evaluation of students does take place at the work-site. However, 
rather than evaluating work-based skills attained by students in relationship to an 
individualized training plan, the evaluation is often performed in accordance with 
standards set for all employees of the business. Evaluation of work-based skills attained 
by students was not usually conducted nor did the work-based learning coordinators and 
employers conduct performance reviews. Also, performance reviews are seldom 
conducted Jointly between employers, work-based learning coordinators, and students. 

Workplace mentor or training supervisor. Within a business or industry, there is a 
single individual who has agreed to serve as the student’s mentor or training supervisor 
and is responsible for the day to day instruction of the student while on-the-Job. The 
mentor or training supervisor should be willing to work with the work-based learning 
coordinator in planning on-the-Job learning experiences and related classroom instruction 
and in evaluating student progress. 

Data indicated that a majority of employers identified an individual to serve as a 
student’s mentor or training supervisor, but few had been appraised as to their 



responsibilities to either the student or the program. Very few mentors or supervisors had 
attended or participated in formal workshops or orientation meetings. 

Supervisory (Coordination) visits. One of the responsibilities of the work-based 
learning coordinator is to make each student’s on-the-job training an effective learning 
experience. This is accomplished through a system of regularly scheduled supervisory 
(coordination) visits to each employer’s business in which students are placed for on-the- 
job instruction. 

Data implied that there were initial visits by the work-based learning coordinator to 
introduce themselves and the program but few formal visitations once students were 
employed on the job. However, the employers reported frequent informational 
communication vvith the work-based learning coordinator. 

Curriculum. The curriculum is the related in-school instruction which students in a 
work-based learning program receive in conjunction with their on-the-job training. The 
curriculum should be related to the job placement of the student. 

Data provided by work-based learning coordinators of work-based learning programs 
indicated that the curriculum followed in each program came from curriculum guides and 
textbooks from various sources. The curriculum followed by the work-based learning 
coordinators specified certain units and objectives and very limited flexibility. 

There is a limited relationship between the curriculum and instructional activities that 
students experience in the school and the activities and tasks that they perform on the 
work site. Additionally, further student data tends to reflect the lack of integration of 
academic and vocational subject matter that would allow a connection to work-based 
experiences and the school-based curriculum. 

Process Issues 

The following process issues for work-based learning programs were identified 
from data collected from the three sup-populations of this study. 

Career awareness and development. Giving direction to the career development 
needs of students is an important aspect of a work-based learning program. Based on 
data provided by the three sup-populations, programs need to strengthen career awareness 
and exploration activities available to students. Participation in such activities could 
serve to validate the students’ career goals and their related placement in a work-based 
learning situation that would be more appropriate to the attainment of their goals. 

Promotion of the program. Promotional marketing efforts should have two 
functions: (1) to provide information that shapes the way the work-based learning 
program is perceived by individuals in the school, parents, and the community at large; 
(2) to create a pool of applicants for the student selection process. Based on results of 





this study, marketing efforts in work-based learning programs vary in intensity; most are 
minimal doing just enough to fill the required openings for entering students. 

Articulation. A work-based learning program should seek to increase students’ 
opportunities to pursue post secondary education or advanced training. The recent 
movement towards linkages of secondary and post secondary education programs was 
not evident in the data on work-based learning programs, yet a large majority of students 
participating in the programs were planning to enroll in some type of post secondary 
institution or training program upon completion of high school. 

Linking school-based learning with work-based learning. As noted in the 
curriculum findings, the linkage between what a student is learning in the school and 
what the student is doing on the Job is somewhat limited in most instances. Few 
employers in this study participated in the conventional aspects of work-based learning 
programs such as training agreements, training plans, and student evaluation. Thus there 
may be limited linkage between what students learn at school and at work, because the^ 
employers are unaware of the expectations of the program including the curriculum. 

Program evaluation. To determine the effectiveness of a work-based learning 
program, an evaluation model should document both the short-term benefits and long- 
term consequences of participation in the program. Data suggested that a formative 
evaluation process was conducted in only a small number of programs and that the results 
of these limited evaluations were underutilized. 

Overall Conclusions and Recommendations 

Beyond helping students find part-time jobs, work-based learning programs 
should help prepare them for future career mobility and further education. The following 
could enhance work-based learning programs within the state of Georgia: 

1 . Increasing the involvement of business and industry personnel within the program. 

2. Developing effective secondary to post secondary linkages, including at least one 
year of post secondary education. 

3. Implementing career exploration and information covering an array of opportunities 
and exposure of students to all aspects of an industry; 

4. Augmenting the structural aspects of the programs including, but not limited to, 
student identification. Job placement, training agreements, training plans, mentor or 
training supervisor development, supervisory visitations, and student evaluation; 

5. Strengthening the programs of study, including both curriculum and instruction. The 
integration of academic and vocational learning as well as school-based and work- 
based learning should be documented. 

6. Improving both the marketing activities and program evaluation processes of all 
work-based learning programs. 



References 



National School-to-Work Office (1997). School-to-work glossary of terms. 
Washington, DC: Author. 

School-to-Work Opportunities Act (1994). US Congress. 

Smith, C.L., & Rojewski, J.W. (1993). School-to-work transition: Alternatives 
for educational reform. Youth and Society, 25 (2), 222-250. 

Smith, C.L., & Thorton, G. (1998a). Assessment of work-based learning 
programs in the secondary schools of Georgia. Athens, GA: University of Georgia. 

Smith, C.L., & Thorton, G. (1998b). Student assessment of school-sponsored 
employment programs in Georgia. Athens, GA: University of Georgia. 

Smith, C.L., & Thorton, G. (1998c). Employer involvement in school-sponsored 
employment programs in Georgia. Athens, GA: University of Georgia. 

Stem, D., Hopkins, C., Stone, J.R., & McMillion, M. (1990). Quality of students 
work experiences and orientation toward work. Youth and Society, 22 (2), 263-282. 




205 

191 



ASSESSMENT IN THE VOCATIONAL EDUCATION CLASSROOM: USING 
INFORMATION FOR INSTRUCTIONAL DECISIONS 



Howard R. D. Gordon, Professor 
Marshall University 

Department of Adult and Technical Education 

400 Hal Greer Boulevard 

Huntington, WV 25755-2460 

(304) 696-3079 

e-mail: gordon@marshall.edu 



Richard J. Yocke, Regional Teacher 
Educator 

West Virginia University Institute of 
Technology 

Dept, of Vocational-Technical Education 

3414 Omdorff Hall 

Montgomery, WV 25136 

(304) 522-8973 

e-mail: rvocke@ezwv.com 



Abstract 

Secondary vocational education teachers were more likely to use information from performance 
assessments the most in making instructional decisions. More than one-half of the respondents 
reported to be very competent in the assessment process. A majority (over three-fifths) of the 
respondents reported a positive attitude toward assessment. Secondary vocational education 
teachers were more likely to decide what assessment methods to use. However, secondary 
vocational education teachers lacked planning time to use assessment methods effectively. 
Secondary vocational education teachers were less likely to use the following methods of 
assessment: portfolios, essay-type items, and standardized tests. These methods accounted for a 
small percent of the variance with selected independent variables (competence, attitude, and 
level of constraint). To be effective users of multiple assessments, secondary vocational 
education teachers need to be familiar with the strengths and weaknesses of different assessment 
methods. 



Introduction 

Over the years, there have been periodic education reform movements. The latter half of 
this century has seen increased emphasis on mathematics and science, prompted by the Soviets’ 
launching of Sputnik; the movement to create and expand community colleges; the movement 
toward (and later retreat from) open schools and classrooms; increased attention to the needs of 
special education population students; generated by the broad social movement of the 1 960s and 
1970s; and more recently, the calls for better education to improve the nation’s competitiveness 
in the global economy. 

Recent education reforms began in the early 1980s and have focused on secondary 
education, prompted by concern about the nation’s declining competitiveness in the international 
market, the relatively poor performance of American students on tests of educational 
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achievement (both nationally and internationally), and complaints from the business 
community about the low level skills and abilities found in high school graduates entering the 
workforce. 

The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 required 
accountability of all states that accept federal funds to support vocational education programs. 
This was to be achieved through a system of specified performance measures and standards 
which track both academic and occupational competency gains. 

Pressures from all levels of policy making — from local school boards to Congress — for 
higher academic and skill standards, more accountability, and better certification of what 
students know, have led to a flurry of activity in the realm of assessment. Teachers are 
experimenting with a variety of tests — and spending a lot more time giving them (Hoachlander, 
1998). 

Virtually all states employ some form of statewide assessment, and many are testing at 
more grade levels and in more subjects than ever before. The Atlanta-based High Schools That 
Work Consortium has embraced rigorous and frequent assessment as part of a commitment to' 
keep score and promote school improvement, as have several other education reform groups 
(Hoachlander, 1998). 

Accompanying this increase in testing activity is a push for different kinds of tests — 
more “authentic” assessment than that produced by the traditional multiple choice exams and 
other forms of paper-and-pencil tests that have been the mainstay of assessment during the past 
three decades. Portfolios, student exhibitions, performance events, simulations, and tests that 
employ “open” or constructed responses all have found their way into the lexicon and practice of 
assessment. 

How are secondary vocational education teachers using assessment data? The literature 
has revealed very little information about the assessment practices of this group of teachers. 
Given the importance of accountability in vocational education, we do need to know more about 
how vocational education teachers use assessment information. 



Related Literature 



Assessment is a broad term defined as a process for obtaining information that is used for 
making decisions about students, curricula and programs, and educational policy (see American 
Federation of Teachers, National Council on Measurement in Education, and National Education 
Association, 1990). 

Decisions about students include managing classroom instruction, placing students into 
different types of educational programs, assigning students to appropriate classifications, guiding 
and counseling them, selecting them for educational opportunities, and credentialing and 
certifying their competence. Decisions about curricula and programs include decisions about 
their effectiveness and about ways to improve them (Nitko, 1996). 

Kershaw and McCaslin (1995) cite six general categories as representing the various 
assessment methods which secondary vocational education teachers use to generate information 
for decision making. The categories of assessment methods revealed were (1) objective paper- 
and-pencil items, (2) standardized test scores, (3) informal observations, (4) essay-type items, (5) 
portfolios and performance assessments (a student’s performance on homework, lab work, 
research papers, and projects). Gullickson (1984) reported that teachers viewed tests as important 
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instructional tools with the time and effort required for their use. Participants in the study 
agreed that tests increase student effort, affect student self-concept, create competition, improve 
student interaction and, in general, improve the learning environment. Green and Stager (1986) 
determined that a relationship of practical significance between attitudes and test use did not 
exist. They concluded that the relationship was probably affected by numerous other factors 
such as accountability demands, students’ expectations and expectations of peers. 

Hall, Carroll, and Comer (1988) investigated the relationship of assessment use to degree 
of autonomy in teaching. Their study revealed that the more autonomous teachers were, the less 
inclined they were to use the results of tests that had been imposed upon them. The authors 
concluded that autonomy potentially serves as a mediating factor in the application of test results 
in the classroom. 

Gullickson and Hopkins (1987) concluded that the substantial constraints imposed on 
educational measurement courses will continue to cause students to.be inadequately prepared for 
classroom evaluation tasks. Strong differences exist regarding what measurement instruction 
pre-service teachers should receive. “Teachers have not learned, and rarely apply those concepts 
that apparently receive major emphasis in measurement instruction” (p. 12). 

Newman and Stallings (1982) reported that four of the six purposes for testing — 
assessing achievement, assessing, mastery, assigning grades, and planning instruction were 
related to a teacher’s knowledge of testing. Gullickson (1984) stated that “the average teacher 
does not perceive college courses to be pertinent to his/her classroom testing needs” (p. 245). 
Although teachers appeared to be comfortable in their knowledge of assessment, Gullickson and 
Hopkins (1987) perceived that they were “much less prepared than what is desirable” (p. 245). 
Hills (1991) indicated that the most serious problem with teacher competence in this area was the 
fact that the primary means of assessment, the classroom test, is “often seriously flawed or 
misused” (p. 542). 

The review of the literature reveals that limited studies have been conducted to address 
the perceived use of assessment methods by secondary vocational education teachers. The 
results of this study should be useful to the body of knowledge regarding the assessment 
practices of secondary teachers in West Virginia’s vocational education programs. 



Purpose and Objectives 

The purpose of this study was to describe West Virginia secondary vocational education 
teacher’s use of student assessment information in making instructional decisions. The specific 
objectives of study were as follows: 

1 . To describe secondary vocational education teachers’ perceptions of their use of 
student assessment data for making instructional decisions. 

2. To describe secondary vocational education teachers’ perceptions of their 
competence, constraints, and attitudes toward the assessment process. 




194 



208 



3 . 



To determine what relationships exist between teacher use of assessment 
information and selected independent variables (competence, attitude, and level 
of constraint). 



Procedures 

The target population (N = 647) was all teachers in West Virginia who taught full-time at 
secondary vocational technical centers during the 1997-98 school year. The 1997- 98 West 
Virginia Education Directory was used to identify the population, which served as the sampling 
frame for the study. Using Krejcie and Morgan’s (1970) table of sample sizes, a sample size of 
240 was identified as representative of a population of 647 within a five percent margin of error. 

A cluster sampling technique was used to randomly select secondary vocational education 
teachers from the 32 secondary vocational technical centers in the state of West Virginia. 
Twelve secondary vocational technical centers were randomly selected to participate in the study 
in order to achieve the desired sample size of 240 (12 schools vvdth an average of 20 teachers per ~ 
school). Secondary vocational teclmical centers were numbered from 1 to 32, and the 12 schools 
were selected using the random number generation in Microsoft Excel. 

Cluster sampling is typically used when researchers cannot get a complete list of 
members of a population they wish to study but can get a complete list of groups of “clusters” of 
the population. It is also used when a random sample would produce a list of subjects so widely 
scattered that surveying them would be prohibitively expensive (Vogt, 1993). However, the 
disadvantage of cluster sampling is that each stage of the process increases sampling error. 



Instrumentation 

The five-part survey instrument used in this study was adapted from a study conducted by 
Kershaw and McCaslin (1995). The use of assessment information was measured in part one of 
the instrument. Participants were asked to indicate the extent to which they use the information 
derived from six types of assessment methods in addressing 10 different instructional decisions. 
The six types of assessment methods used in this study were (1) objective paper and pencil 
items, (2) informal observations, (3) standardized test scores, (4) performance assessments, (5) 
portfolios, and (6) essay type items. The 10 instructional decisions ( Kershaw & McCaslin, 
1995; Nitko, 1996) that were addressed were (1) planning instruction, (2) diagnosing student 
weaknesses, (3) monitoring student progress towards course objectives, (4) communicating 
student achievement with parents, (5) motivating students to learn, (6) evaluating the 
effectiveness of instruction, (7) evaluating the instructional materials used, (8) grouping students 
for instructional activities, (9) encouraging students to assess their own work, and (10) assigning 
grades. Items were scored on a 5-point Likert scale ranging from “of no use” (1) to “of 
considerable use” (5). 

Part two of the instrument measured participants’ perceived level of competence in the 
assessment process. Participants were asked to indicate their competence in the assessment 
process using a series of competency statements based on “ Standards for Teacher Competence 
in Educational Assessment of Students” (American Federation of Teachers, National Council on 
Measurement in Education, & National Education Association, 1990). Items were scored on a 
5-point Likert scale ranging from “ not competent” (1) to “extremely competent” (5). 
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Attitudes toward assessment were measured in part three of the questionnaire using a 
semantic differential scale. The scale was comprised of nine bi-polar adjectives which described 
the concept “ assessment.” A seven-point scale was used for each adjective pair to describe their 
attitudes toward the overall assessment process. 

In part four of the questionnaire, nine statements were used to measure teacher’s 
perceptions of constraints to their assessment activities. The 5-point Likert scale used in this 
section ranged from “strongly disagree” (1) to “strongly agree” (5). Part five of the instrument 
consisted of demographic questions. 

To ensure validity of the instrument, a panel of experts was used to establish content and 
face validity. The panel consisted of three vocational education teachers, a regional teacher 
educator and two professors of vocational education. The instrument was field-tested for 
reliability with a sample of secondary vocational education teachers ( n = 1 1) not selected for 
participation in the study. Changes indicated by the validation panel and field test were made. 
Internal consistencies for the scales in the instrument were as follows (Cronbach’s alpha); Use of 
Assessment .94, Competency in Assessment .93, Attitudes Toward Assessment .95, and — 
Constraints in Assessment .64, acceptable according to Nurmally and Bernstein (1994). 



Data Collection 

The Total Design Method (TDM) of conducting surveys (Dillman, 1978) was followed in 
all stages of the questionnaire construction and implementation process. A packet containing a 
cover letter, instructions for administering the questionnaire, and copies of the questionnaire 
were mailed to the principal of each school selected for the study. A total of 240 questionnaires 
was sent to 12 principals during October, 1997. A total of 144 usable questionnaires was 
returned for a response rate of 60%. Because a number of questionnaires were returned 
uncompleted and there was no way to conduct appropriate follow-up procedures to control for 
non-response, the results of the study can only be generalized to the 144 teachers who provided 
usable data. 



Data Analysis 

Data were analyzed using the Statistical Package for the Social Sciences (SPSS Version 
6.1 for Windows). Descriptive statistics were used to summarize the data. Pearson’s product- 
moment correlational analyses were conducted to determine relationships between use of 
assessment information and selected independent variables. The following standards as 
presented by Best (1981) were used to interpret the correlation coefficients (negligible = .00 to 
.20; low = .20 to .40; moderate = .40 to .60; substantial = .60 to 80; high to very high = .80 to 
1 . 00 ). 
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Results 



Demographic Characteristics 

The largest number of teachers in the sample fell within the 42-51 year age bracket 
(52.1%) of the 144 cases, 56% of the respondents were male and 44% were female. Teachers had 
an average of 15.02 years of teaching experience and 9.66 years of related work experience. 
Trade and industrial teachers comprised the largest group in the sample and accounted for 43% 
of the cases. A graduate degree had been earned by 39% of the respondents. Respondents, who 
had completed a teacher preparation program on the job, and before receiving a baccalaureate 
degree, comprised 43% of the cases. 

Teacher’s Use of Assessment Information 



Table 1 depicts teachers’ use of assessment information and educational decision areas. 
Information generated from performance assessment was considered to be of more use to 
teachers in addressing instructional decisions than the other five assessment methods (M = 4 
.19). Teachers rated performance assessment information to be of much use (M = 4 .45), when 
specifically addressing the task of assigning grades. Teachers rated information derived from 
informal observations to be more useful than the other assessment methods when grouping 
students for instructional activities ( M = 3. 95 ). Portfolios, standardized test scores, and essay 
items were found to be of less importance to secondary vocational education teachers when 
compared to the other methods. 



Means^ and Standard Deviations for the Use of Assessment Information Generated From Six Assessment Methods (n_= 144) 
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Perceived Level of Competency in Assessment 



Table 2 provides information regarding teachers’ perceived level of competence in 
individual assessment activities. Teachers considered themselves to be very competent (M = 4 
.35 - 3 .52) in all but three areas of the assessment process (evaluating a student portfolio, 
grading an essay, and interpreting the scores of standardized tests. 

Table 2. 



Means° and Standard Deviations for Teachers Perceived Competence in the Assessment Process 



(n=144)'’ 


Characteristics 


M 


SD 


Grading objective paper and pencil items 


4.35 


.76 


Grading performance assessment 


4.14 


.82 


Prepare students to take tests 


4.12 


.74 


Administering a performance assessment 


4.11 


.85 


Commimicating assessment results to students 


4.07 


.85 


Developing student grading procedures 


4.06 


.86 


Selecting methods for assessing student performance 


4.05 


.79 


Matching items to intended learning outcomes 


4.03 


.83 


Selecting assessment methods for monitoring student learning 


3 .93 


.76 


Commimicating assessment results to employers 


3 .84 


.84 


Using assessment results to monitor student learning 


3 .83 


.90 


Recognizing unethical methods of using assessment 


3 .80 


1.04 


Determining proper difficulty of items 


3 .78 


.89 


Developing a performance assessment rating scale 


3 .73 


.90 


Identifying the weakness of assessment methods 


3 .73 


.74 


Using assessment results to organize a sound instructional plan 


3 .73 


.93 


Writing directions for assessment methods 


3 .72 


.94 


Validating of test items 


3 .70 


.90 


Selecting a representative sample of items for use 


3 .70 


.90 


Communicating assessment results to parents 


3 .67 


1.03 


Determining appropriate number of items for assessment 


3 .67 


.86 


Identifying structural problems in objective questions 


3 .62 


.84 


Examining items for gender bias 


3 .52 


1.00 


Evaluating a student portfolio 


3 .38 


1.15 


Grading an essay 


3 .36 


1.05 


Interpreting the scores of standardized achievement tests 


3 .24 


1.16 



Note. ° Based on scale : 1= not competent; 2 = slightly competent; 3 = moderately competent; 4 = 
very competent; 5 = extremely competent. 

Three missing cases. 
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Attitudes Toward Assessment 



A semantic differential scale was used to measure teacher’s attitudes toward the overall 
assessment process. Teachers were asked to respond to nine pairs of bi-polar adjectives which 
were on either end of a 7-point scale. The scale was as follows: 1 = extremely negative; 2 = very 
negative; 3 = negative; 4 = neutral, 5 = positive; 6 = very positive; 7 = extremely positive. 
Overall, there was no indication that a negative attitude (M < 4 .0) existed towards the 
assessment process. 

Constraints to the Assessment Process 



Table 3 describes the responses to each of the constraint statements. Teachers tended to 
agree that they decided on what assessment methods to use in their courses (M = 4 .28). Teachers 
also tended to agree that additional planning time would allow for assessment methods to be 
used more effectively (M = 3 .94). Over two-thirds of the items were rated as “neutral” (neither 
agree or disagree) by respondents in this study (M = 2 .69 - 3 .00). 
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Tables. 



Process Tn = 144) 


Item 


M 




I decide what assessment methods 
to use in courses I teach 


4.28 


.97 


Additional planning time would allow me to use 
assessment methods more effectively 


3 .94 


1 .10 


Equipment is available in my school for use 
in scoring tests 


3.06 


1 .41 


Quality published assessment materials are hard 
to find 


3 .00 


1 .18 


Funds are available for buying published assessment 
materials 


2.91 


1 .32 


College courses were of little help in preparing me 
to assess student learning 


2.86 


1 .25 


In-service activities have helped develop 
my assessment skills 


2.81 


1 .15 


I have assistance in preparing student assessment 
activities 


2.75 


1 .37 


I do not have information on published assessment 
materials 


2.69 


1 .20 



Note . “ Based on scale: 1 = strongly disagree; 2 = disagree; 3 = neither agree or disagree; 4 = 
agree; 5 = strongly agree. 
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Relationships Between Selected Independent Variables and Assessment Methods 

The relationships between the use of assessment information and selected independent 
variables in the study are illustrated in Table 4. Teachers’ perceived level of competence in 
assessment was shown to have a low positive relationship with the use of information from 
objective paper and pencil methods (r = .33, r^ = . 1089). 

Attitudes toward assessment had a low and positive relationship with teachers’ use of the 
assessment information derived from performance assessment (r = .35, r^ = .1225). Constraints to 
the assessment process had a low and positive relationship with the use of performance 
assessment (r = .21, ? = .0441). Use of standardized tests had a negligible association with 
competence in and attitude towards assessment. The data also revealed that essay items and 
portfolios had a negligible association with attitude towards the assessment process. 

Table 4. 

Intercorrelations Between Selected Independent Variables and Assessment Methods (n = 144) 



Variable 1 2 


3 


4 


. 5 


6 


7 


8 


9 


1. Competence — .37 


.12 


.33 


.12 


.27 


.31 


.18 


.25 


2. Attitude 


o 

r 


.29 


-.05 


.35 


.31 


.01 


.03 


3. Level of Constraint 


— 


.12 


.18 


.21 


.09 


.20 


.14 


4. Objective Item 




— 


.30 


.29 


.27 


.27 


.34 


5. Standardized Test 






— 


.06 


-.07 


.41 


.49 


6. Performance Assessment 








— 


.60 


.19 


.09 


7. Informal Observation 










— 


.11 


.10 


8. Portfolio 












— 


.57 


9. Essay Item 














— 



Note. Correlations above .16 are statistically significant at the .05 level. 



217 

202 



Discussion and Conclusions 



Results of this study revealed that performance assessments are particularly useful to 
secondary vocational education teachers. In previous research, Kershaw and McCaslin (1995) 
found that secondary vocational education teachers tended to place more emphasis upon the use 
of performance assessments than on other methods. Given that vocational education teachers use 
a competency based curriculum, it was not surprising to the researchers that a performance based 
method of assessment was of more use than the other five methods of assessments. Performance 
assessments has always been prominent among T & I teachers, which compose much of this 
sample. This is not a new phenomena among all secondary vocational education teachers. 

Secondary vocational education teachers in this study reported that they perceived 
themselves to be moderately to very competent in the assessment process. The data seem to 
indicate that secondary vocational education teachers demonstrate a general level of competence 
in assessment, but do not feel as competent when dealing with assessing portfolios, essay items, 
and standardized tests. These results were consistent with findings from previous studies 
(Gullickson & Hopkins, 1987; Kershaw & McCaslin, 1995). 

A majority (66%) of the respondents reported a positive attitude toward assessment. 
However, this was probably due to the fact that teachers in this sample were mostly 
homogeneous. On the other hand, this finding mirrors previous studies done by Green (1990) 
and Kershaw and McCaslin (1995). 

Secondary vocational education teachers in this study were more likely to decide what 
assessment methods to use. However, secondary vocational education teachers lacked planning 
time to use assessment methods effectively. In a study by Griswold (1988), teachers reported 
that they required more time if meaningful assessments were to be developed or improved. 

Coefficients of determination revealed that use of portfolios, essay items, and 
standardized tests accounted for a small percent of the variance with selected variables 
(competence, attitude, and level of constraint). This finding suggests that secondary vocational 
education teachers lacked the necessary skills to use these three methods of assessments. This 
finding is consistent with previous studies in assessment use (Goslin, 1967; Yeh et al., 
1981;Green, 1990; Kershaw and McCaslin, 1995). 

Overall, the results of this study seem to suggest that secondary vocational education 
teachers are doing an adequate job (passable but not outstandingly good) concerning the use of 
assessment for instructional decisions. 
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Recommendations 



The following recommendations are based on the findings and conclusions of this study. 

1. To be effective users of multiple assessments, secondary vocational education 
teachers need to be familiar with the strengths and weaknesses of different 
assessment methods. Each method should be selected to achieve a well- 
considered objective and the complete set of strategies designed to provide a well- 
rounded summary of what students know and can do. 

2. Pre-service and in-service providers should structure workshop to address the 
necessary skills in evaluating a student portfolio, how to evaluate essay test items, 
and how to interpret the scores of standardized tests. 

3. To make changes that are conceptually meaningful, secondary vocational 
education teachers need: appropriate materials to try out and adapt; time to reflect 
and to develop new instructional approaches, and ongoing support from experts. 

4. Qualitative studies should be conducted to examine the use of assessment 
methods by vocational education teachers. Qualitative research involves intensive 
data collection, that is, collection of extensive data on many variables over an 
extended period of time, in a naturalistic setting. Variables which contribute 
additional variance in the use of assessment information could be identified and 
examined. 
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MINI-TIP 



Using Qualitative Research to Frame the Examination 
of an Alternative Vocational Teacher Education Program 



Gloria Heberley, Field Resource Associate 
Temple University 
Vocational Education/CITE 
Ritter Hall 354 
Philadelphia, PA 19122 

Phone (215) 204-8374 FAX (215) 204-5154 
e-mail Heberley @vm.temple.edu 



The presentation will discuss the results of doctoral research which investigated a 
structure for use in the examination of an alternative vocational teacher education program. 

The objective of the research was to verify the effectiveness of a previously identified 
cluster of common elements in a qualitative illumination of an alternative vocational pedagogical 
delivery system. Selected findings reveal the views of faculty and students on the program goals, 
identification with the program, the program structure, research on teaching, and students’ 
passage through the program. 

The conclusions authenticate the value of the structure for use in scrutinizing existing 
traditional and alternative teacher education programs. Further, the framework can be used to 
assess, transform, or initiate such programs. 
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MINI-TIP 

CHARACTERISTICS OF A CONCEPTUAL FRAMEWORK ADDRESSING THE 
ENVIRONMENT OF THE INSTRUCTIONAL SPACE-HOLISTICALLY; DESIGN USERS 
PEDAGOGY, AND TECHNOLOGY. 

By: Dr. Lennie Scott-Webber, (1998) Assistant Professor — ^Interior Design, Virginia 

Polytechnic Institute and State University; 213 Wallace Hall, Blacksburg, VA 24060 — 
0410. 




USER’S ENVIRONMENTAL INTERACTION FRAMEWORK® 



This research studied the classroom environment as a holistic setting: user, environment, 
proxemics, pedagogy, and technology. The conceptual framework “User’s Environmental 
Interaction Framework (Scott-Webber, 1998) shown above is divided into three distinct areas. 
Aea I includes environmental and value dimensions along with internal and behavioral 
responses from the subject (user); analyzed by a questionnaire format. Aea II, the proxemic 
zones (Hall, 1969) studies user behavioral activities in these zones — by mapping techniques. 
The third area includes interactions imposed upon the space; in this case, pedagogy and 
technology — using both questionnaires and mapping techniques. Step-wise and Canonical 
Discriminant Aalyses were used to analyze the 52 questions administered to 120 subjects. 
Overall, there was significance between faculty and students in environmental and value 
dimensions, and internal responses, but not with behavioral responses. The mapping signified 
the majority, of faculty, continue to keep their backs to students and write on the blackboard. 
Students are grouped too closely together (encroachment on the intimate zone), and feel 
crowded. While some faculty stmggled against their environmental settings (e.g., a rectangular 
box) and tried different pedagogical styles, most indicated that the setting forced them to return 
to the lecture format (e.g., fixed seating or large class sizes). Technology was used in the 
classroom primarily to support the lecture format; presentations were given using PowerPoint 
software or the television to present a movie, or video information. The conceptual framework 
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proved a useful tool ensuring all aspects of a setting are considered in researching 
human/environment interactions. 

Reference: Hall, ET (1969). The hidden dimension. New York: Doubleday. 
Scott-Webber, L. (1998). Programming: A problem solving approach for users of interior 
spaces. Houston, TX: Dame Publishers Inc. 




MiniTip 

Strengths and Weaknesses of E-mail as a Survey Research Method 

Dr. Allen D. Truell 
University of Missouri-Columbia 
316 Hill Hall 
Columbia, MO 65211 
Phone: (573) 882-9619 
Fax: (573) 882-5071 
E-mail: Davtat@tiger.coe.missouri.edu 

Purpose : This session will report on the recent developments associated with using e-mail as a 
survey delivery and collection technique. During this session, I will report on the strengths and 
weaknesses of using e-mail as a survey research tool in a variety of settings. 

Description : The strengths and weaknesses of e-mail as a survey research tool follows: 

Strengths 

1 . Delivery/response speed-surveys can be delivered or redelivered anywhere in the world 
in a matter of seconds. 

2. Low costs— the e-mail method of data collection costs less than face-to-face interviews 
and mail and telephone surveys. 

3. Geographic coverage-e-mail surveys can be delivered to locations around the 
community or around the globe with ease. 

4. Favorable response rates— e-mail survey response rates compare favorably with those of 
other survey research methods. 

5. Ease of editing-being computer based, e-mail surveys and participant mailing lists are 
easily edited. 

6. Open responses— researchers have found participants to be more open when responding to 
an e-mail survey than those individuals replying to a paper and pencil survey. 

7. Environmental correctness-e-mail surveys reduce the need for envelops, ink cartridges, 
and paper which lessens the demand for natural resources. 

8. Semi-interactive— being semi-interactive, participants are able to communicate with the 
researcher if clarification is needed. 

9. Response options— participants have several methods of returning the survey: e-mail, fax, 
and traditional postal mail. 

Weaknesses 

1. Technical problems— due to incompatible hardware and software, many technical 
problems may occur. 

2. Reduced confidentiality-participants may be reluctant to provide honest responses 
because of the perceived lower level of confidentiality associated with e-mail. 
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3. Sample selection-population and sample selection is limited to those individuals who 
have and use an e-mail account. 

4. Supplemental orientation— additional instruction may be required for participants to 
completed and return the survey online. 

5. Presentation difficulties-hardware and software limitations may make developing a high- 
quality survey presentation difficult. 

6. Bias problems— self-selection bias is likely to taint the results of an e-mail survey. 

A detailed paper outlining the strengths and weaknesses of using e-mail as a research tool will be 
provided to participants at the presentation. Additional information can be obtained by 
contacting Dr. Allen D. Truell, University of Missouri-Columbia, 316 Hill Hall, Columbia, MO 
65211 or calling (573) 882-3296 or e-mailing pavtat@tiQer.coe.missouri.edu . 



Justification for Grand Carousel Presentation 

Like other survey research methods that preceded it-face-to-face interviews and mail and 
telephone surveys-e-mail research is evolving. E-mail presents a number of distinct advantages 
for survey researchers; among the most commonly noted are speed, lower costs, wide geographic 
coverage, favorable response rates, ease of editing, openness of responses, environmental 
correctness, semi-interactive, and a variety of response options. The most commonly reported 
weaknesses of e-mail are technical problems, reduced confidentiality, sample selection, need for 
supplemental instructions, presentation difficulties, and bias problems. As more studies are 
conducted, researchers may smooth out these rough spots associated with e-mail use; however, 
researchers exploring the use of e-mail must always evaluate its strengths and weaknesses as a 
data collection method in a specific context. All indications are that the use of e-mail as a survey 
research tool will continue to increase. 
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Mini Tip 



A Redesigned Program for the Preparation of Teachers 



Bob R. Stewart, Associate Director 
Undergraduate Teacher Development 
Professor and Chair, Council of 
Vocational-Technical Education 
University of Missouri-Columbia 
323 Townsend Hall 
Columbia, Missouri ,65211 
573-882-8391 



Martin Bergee, Director 
Undergraduate Teacher Development 
Associate Professor and Chair, 
Curriculum and Instruction 
University of Missouri-Columbia 
212 Townsend Hall 
Columbia, Missouri 65211 
573-882-6572 



Much attention is being focused on restructuring the preparation of teachers for our nation’s 
schools. Professional development schools, university-school partnerships, field-based 
experiences, contextual learning, reflective teaching, and problems of practice are all concepts 
associated with the preparation of teachers. The session will present the design for a redesigned 
teacher preparation program using a university-school partnership model to involve students in 
an integrated field experience component each of their last six semesters of coursework. In 
addition, the overall design of the program will be presented as well as the plan for faculty 
input/govemance, the role of the partnership in providing field experiences, and a description of 
an induction year teaching fellows program designed to provide mentoring for new teachers and 
inservice activities for teachers in the school. 

Both the academic and popular press have focused on the need for teacher preparation 
programs to be redesigned. Considerations involve changing the culture of public and private 
schools at the same time that colleges and departments of education are changing in approach 
and delivery. Earlier and sustained field experience are key features of such proposals. Other 
issues involve faculty input/govemance, induction year experiences, and the length of the 
teaching internship. 

Participants will be able to review: the initial lock step four year program design with an 
immersion semester option as compared to the redesigned three year sequenced program with an 
immersion option; the structure and responsibilities of the faculty committee charged with 
oversight of the program; and, the role of the school-university partnership in coordinating the 
integrated field experience phase of the prograni. 

It was concluded that evaluation and reflection are key components of redesigning teacher 
preparation programs. Faculty, partnership teacher, and student input are all important in 
implementing and sustaining program change. 



227 

o 

ERIC 



212 



Mini Tip 



Exploratory Style of Vocational Educators 

Helen C. Hall, Ph.D. or Bettye P. Smith, Ph.D. 

The University of Georgia 
Department of Occupational Studies 
216 River's Crossing 
850 College Station Rd. 

Athens, GA 30602 



O 

ERIC 



Purpose and/or Objectives of the Presentation 

The objectives of this study were to examine the explanatory style of secondary 
vocational educators and determine their ratings based on: 1) composite positive, composite 
negative, and composite positive minus composite negative; and 2) personalization, 
permanence, and pervasiveness. 

Brief Description or Discussion of Topic 

Explanatory style is one's habitual way of explaining events, good and bad; it is a habit of 
thought. Explanatory style is a descriptive term used to explain variations in people's response to 
uncontrollable events. It illustrates how people habitually explain good and bad events they 
encounter in life; it is the hallmark of whether you are an optimist or a pessimist. Explanatory 
style reflects individual differences along three dimensions. Those dimensions are permanence, 
pervasiveness, and personalization. Permanence is about time; it determines how long a person 
gives up (stable or unstable). Pervasiveness is about space, global and specific. People who 
make universal explanations for their failures give up on everything when a failure strikes in one 
area. People who make specific explanations may become helpless in that one part of their lives. 
Personalization is the easiest to understand. When bad things happen, we can blame ourselves 

(internalize) or we can blame other people or circumstances (externalize). 

228 

213 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 



® 







REPRODUCTION RELEASE 

(Specific Document) 



1. uuuun 
“ 1 


ncni 1 1 


Aiithr^rfsV / U/\ 1 H P 


— = — / 

Corporate Source: 

M \/^€4 


Publication Date: 



II 

■ I* 



ocpoQnijrTiQW p.ELE^.SE: 

In order to disseminate as widely as possible timely and significant materials of interest to the educational communi ^ . documei^ a^^ in the 
monthly abstract journal of the ERIC system. Resources in Educetion (RIE), are usually made available to users in microfic . p P P and^^if 

rd elecSronte media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and. if 
reproduction release is granted, one of the following notices is affixed to the document. 

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 
of the page. 



The sample sdcfcer shown below will be 
■ffivMi to all Level 1 documents 


The sample sticker shown below will be 
affixed to all Level 2A documents 


The sample sticker shown below will be 
affixed to all Level 28 documents 


PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE. AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY. 
HAS BEEN GRANTED BY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTED BY 

/ 


C.'y 








cf. 


TO THE EDUCATIONAL RESOURCES 
information center (ERIC) 




-m/ 

TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


1 




2A 




2B 

1 auaI 9R 



Level 1 Level 2A ^ 



Check here for Level 1 release, pemiittino reproducJion 
and disseminalion in microficha or other ERIC aichtval 
media (e.Q.. eleeironic) and papw copy. 



Check here for Level 2A relaaaa. penrttling repfoducdon 
and dissemination In nrdcroflche and In electronic media 
for ERIC archivai coitectton subscribers only 



Check here for Level 2B release. pennittinQ 
reproduction and dissemination in microfiche only 



Documents wiii be processed as indicated provided reproduction quality permits, 
tf permission to reproduce is Qranted, but no boot is checked, docuntents will be processed at Level 1 . 



Sign 
here,-^ 
O ise 

ERIC 



/ hereby grant to the Educatbnel Resources Infbrmetion Center (ERIC) nonexclusive permission rte^vsfem 

as indicated above. Reproduction from the ERIC microfiche or electronic medie by persons ot^r 

contTBctors requires permission from the copyright holder. Exception is mode for non-profit reproduction by iibranes end Q 

to satisfy information needs of educators in response to discrete inquiries. 




//ie 


Organizattori/Aodresa: 

A i/g/e ^ : 




FIPS'S 


^^ail Address: 

(g? ^ 





(over) 





Hi. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE); 

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source, please 
provide the following information regarding the availability of the document. (ERIC will not announce e document unless it is publicly 
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more 
stringent for documents that cannot be made available through EDRS.) 



Publisher/Distributor 




Address: 




Price; 





IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION R HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 
address; 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinnhouse: , , Cheryl Grossman- . 

Processing Coordinator 

ERIC Clearinghouse on Adult, Career, and Vocational Education 
Center on Education and Training for Employment 
1900 Kenny Road 
Columbus, OH 43210-1090 



However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being 
contnbuted) to: “ 



er|c 



